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Highlights 

Highlights  of  GAO-15-8,  a  report  to  the 
Subcommittee  on  Financial  and  Contracting 
Oversight,  Committee  on  Homeland  Security 
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Why  GAO  Did  This  Study 

The  federal  government  spends 
hundreds  of  billions  of  doliars  annually 
to  acquire  goods  and  services  from  the 
private  sector  to  support  agency 
missions.  Market  research — the 
process  used  to  collect  and  analyze 
data  about  capabilities  in  the  market 
that  satisfy  agency  needs — is  critical  to 
inform  decisions  about  how  best  to 
acquire  these  goods  and  services. 

GAO  was  asked  to  examine  how 
federai  agencies  conduct  and  use 
market  research.  This  report  assesses 
(1)  selected  agencies’  market  research 
guidance  and  policy  and  (2)  how 
selected  agencies  conducted  market 
research  on  selected  procurements. 
GAO  reviewed  federai  regulations  and 
market  research  policy  and  guidance 
at  DOD,  DHS,  and  DOT — three 
agencies  GAO  seiected  based  on 
fiscal  year  2012  contract  obligations 
and  the  mix  of  products  and  services 
procured.  GAO  also  reviewed  policies 
and  guidance  issued  by  the  FAA, 
which  is  not  subject  to  the  Federal 
Acquisition  Regulation.  To  assess  how 
market  research  was  conducted  on 
individual  contracts,  GAO  reviewed  a 
nongeneralizabie  sample  of  28 
contracts  selected  based  on  dollar 
value  and  analyzed  contract  files. 

What  GAO  Recommends 

To  inform  future  acquisitions,  GAO 
recommends  that  the  Secretary  of 
Transportation  update  its  market 
research  guidance  and  the  Secretaries 
of  Defense  and  Fiomeiand  Security 
take  action  to  ensure  their  acquisition 
personnei  cleariy  document  the  basic 
eiements  of  the  market  research  that 
was  conducted.  The  agencies 
concurred  with  these 
recommendations. 

View  GAO-15-8.  For  more  information,  contact 
Timothy  J.  DiNapoii  at  (202)  512-4841  or 
dinapoiit@gao.gov. 
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What  GAO  Found 

Market  research  guidance  at  the  Departments  of  Defense  (DOD),  Homeland 
Security  (DHS),  and  Transportation  (DOT)  and  the  Federal  Aviation 
Administration  (FAA)  is  consistent  with  federal  regulations  in  terms  of  market 
research  objectives  and  builds  on  the  techniques  for  communicating  with  industry 
outlined  in  federal  regulations.  All  four  agencies  require  that  market  research  be 
clearly  documented  and  note  that  documentation  can  inform  current  and  future 
procurements.  GAO  found,  however,  that  the  agencies’  guidance  varied  on  the 
specificity  of  market  research  documentation.  For  example,  DOD,  DHS,  and  FAA 
guidance  identify  specific  market  research  elements  to  be  documented.  Based 
on  analysis  of  these  market  research  elements,  GAO  identified  four  elements 
which,  if  recorded,  would  provide  an  understanding  of  the  research  completed. 
These  elements  include  the  market  research  methods  used,  when  it  was 
conducted,  an  analysis  of  vendor  capabilities,  and  a  conclusion.  DOT’S  market 
research  guidance  did  not  require  specific  elements  to  be  documented,  which 
increases  the  risk  that  it  would  not  be  available  to  inform  future  procurements. 

All  28  contracts  GAO  reviewed  included  some  evidence  of  the  market  research 
conducted.  The  market  research  conducted  on  the  12  higher  dollar  contracts 
GAO  reviewed  tended  to  be  more  robust  and  include  more  techniques  that 
involved  outreach  to  vendors — such  as  issuing  requests  for  information  to 
industry — which  appeared  to  help  promote  competition.  Agencies  did  not  take 
advantage  of  many  available  market  research  techniques  on  the  16  lower  dollar 
contracts  GAO  reviewed  and  as  a  result  may  have  missed  opportunities  to 
promote  competition  (see  figure). 


Figure:  Market  research  techniques  for  28  contracts  GAO  reviewed 
Higher  dollar  contracts  (above  $10  million) 


Lower  dollar  contracts  (between  $500,000  and  $10  million) 
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Degree  of  competition  achieved  Market  research  techniques 
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Source:  GAO  analysis  of  DOD,  DHS,  DOT,  and  FAA  documentation.  ]  GAO-15-8 

GAO  also  identified  limitations  in  the  market  research  for  seven  DOD  and  DHS 
lower  dollar  contracts  that  appeared  to  be  incomplete  or  outdated.  For  example, 
DHS  relied  on  incomplete  information  regarding  potential  vendors’  ability  to  meet 
its  requirement  for  parking  services.  Further,  in  14  of  the  28  contracts,  the  four 
agencies  did  not  document  one  or  more  of  four  basic  elements  that  GAO’s 
review  of  agency  guidance  identified  as  important  to  the  ability  to  understand  the 
research.  GAO  identified  this  shortfall  most  often  on  lower  dollar  contracts 
reviewed  at  DOD  and  DHS.  Internal  control  standards  state  that  significant 
events  need  to  be  clearly  documented  so  as  to  ensure  management  directives 
are  carried  out.  Not  documenting  basic  elements  of  the  market  research 
potentially  limits  the  ability  of  agency  acquisition  personnel  to  use  market 
research  to  inform  future  procurements,  a  goal  identified  in  agency  guidance. 
_ United  States  Government  Accountability  Office 
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U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


October  9,  2014 

The  Honorable  Claire  McCaskill 
Chairman 

The  Honorable  Ron  Johnson 
Ranking  Member 

Subcommittee  on  Financial  and  Contracting  Oversight 
Committee  on  Homeland  Security  and  Governmental  Affairs 
United  States  Senate 

The  federal  government  spends  hundreds  of  billions  of  dollars  each  year 
to  acquire  goods  and  services  through  the  commercial  marketplace  to 
enable  agencies  to  fulfill  their  missions.  These  needs  range  from  simple 
goods  readily  available  in  the  marketplace,  such  as  office  supplies,  to 
services  such  as  janitorial  services,  to  complex  goods  and  services  such 
as  information  technology  systems.  Market  research — the  process  used 
to  collect  and  analyze  data  about  capabilities  in  the  market  that  could 
satisfy  an  agency’s  needs — is  a  critical  step  in  the  acquisition  process, 
informing  key  decisions  about  how  best  to  acquire  goods  and  services. 

For  example,  effective  market  research  can  help  agencies  determine  the 
availability  of  vendors  to  satisfy  requirements;  knowledge  of  marketplace 
suppliers  and  prices  can  be  critical  to  the  government’s  ability  to  negotiate 
a  good  deal  and  foster  competition.  On  the  other  hand,  poor  market 
research  techniques  could  lessen  the  government’s  ability  to  achieve 
competition  and  negotiate  fair  and  reasonable  prices. 

You  asked  us  to  assess  how  federal  agencies  are  conducting  and  using 
market  research.  This  report  assesses  (1)  selected  agencies’  guidance 
and  policy  on  market  research  and  (2)  how  selected  agencies  conducted 
market  research  on  selected  procurements.  To  assess  agency  guidance 
and  policy  on  market  research,  we  reviewed  and  identified  criteria  for  the 
conduct  and  use  of  market  research  in  government-wide  regulations  and 
policy,  such  as  the  Federal  Acquisition  Regulation  (FAR),  including  FAR 
Subpart  7.1 — Acquisition  Plans  and  FAR  Part  10 — Market  Research,  and 
Office  of  Management  and  Budget  (0MB)  memorandums  related  to 
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market  research.''  To  assess  specific  agency  guidance  and  policy  and  to 
illustrate  how  market  research  is  performed  at  agencies  of  various  sizes 
and  that  procure  a  range  of  products  and  services,  we  selected  three 
departments  for  the  review — the  Department  of  Defense  (DOD),  the 
Department  of  Homeland  Security  (DHS)  and  the  Department  of 
Transportation  (DOT).  We  selected  these  departments  based  on  fiscal 
year  2012  contract  obligations — the  most  recent  fiscal  year  of  information 
available  when  we  started  our  review — and  to  get  a  good  mix  of  products 
and  services  procured.  Within  each  department,  we  selected  two 
components  for  further  review — DOD’s  Department  of  the  Army  and 
Defense  Logistics  Agency  (DLA);  DHS’s  Office  of  Procurement 
Operations  (OPO)^  and  U.S.  Customs  and  Border  Protection  (CBP);  and 
DOT’S  Federal  Aviation  Administration  (FAA)  and  Federal  Highway 
Administration  (FHWA) — based  primarily  on  their  fiscal  year  2012 
obligations.  For  each  department  and  component,  we  reviewed  guidance 
and  policy  related  to  market  research,  including  agency  supplements  to 
the  FAR,  market  research  guides  and  templates,  and  policy 
memorandums  that  reference  market  research,  such  as  DOD’s  Better 
Buying  Power  policy  memorandums.  Key  agency  guidance  reviewed 
included  DOD’s  Defense  Standardization  Program,  Market  Research: 
Gathering  Information  about  Commercial  Products  and  Services,  SD-5; 
DHS’s  Market  Research  Guide;  DOT’S  Market  Research  Guide, 
Acquisition  Policy  DOT  Dash  2010-11;  and  FAA’s  Procurement  Guidance 
T3.2.1.2 — Market  Analysis.^  Since  the  FAR  and  DOT  acquisition 
regulation  do  not  apply  to  FAA  acquisitions,  we  analyzed  FAA  policy  and 


^Office  of  Management  and  Budget,  Office  of  Federal  Procurement  Policy,  “Myth-Busting”: 
Addressing  Misconceptions  to  improve  Communication  with  industry  During  the 
Acquisition  Process,  Memorandum  for  Chief  Acquisition  Officers,  Senior  Procurement 
Representatives,  Chief  Information  Officers  (Washington,  D.C.:  Feb.  2,  2011)  and  Office 
of  Management  and  Budget,  Office  of  Federal  Procurement  Policy,  “Myth-Busting  2”: 
Addressing  Misconceptions  and  Further  improving  Communication  During  the  Acquisition 
Process,  Memorandum  for  Chief  Acquisition  Officers,  Senior  Procurement 
Representatives,  Chief  Information  Officers  (Washington,  D.C.:  May  7,  2012). 

^For  the  purposes  of  this  report,  we  considered  DFIS’s  Office  of  Procurement  Operations 
to  be  a  component. 

^While  this  review  focused  on  guidance  applicable  across  each  department  and 
separately  for  FAA,  FFIWA  noted  its  Acquisition  Customer  Service  Guide  contains 
additional  information  on  market  research  techniques. 
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guidance  separately  from  DOT/  We  interviewed  senior  acquisition 
officials  at  each  department  to  develop  a  better  understanding  of  agency 
policies  and  component  guidance  related  to  market  research  and  to 
identify  agency  initiatives  related  to  market  research. 

To  examine  how  agencies  conducted  market  research  on  selected 
procurements,  we  used  the  Federal  Procurement  Data  System-Next 
Generation  (FPDS-NG)  to  select  a  nongeneralizable  sample  of  30 
contracts  awarded  in  fiscal  year  2012  with  contract  dollar  values  of  at 
least  $500,000  from  the  six  components  within  DOD,  DHS,  and  DOT.  We 
selected  this  threshold  as  the  contracts  above  this  threshold  represented 
more  than  92  percent  of  contract  dollars  awarded  in  fiscal  year  2012  at 
each  component  reviewed.  Within  each  selected  agency  component,  we 
created  randomized  lists  of  fiscal  year  2012  contracts  within  two  strata — 
(1)  contracts  valued  at  more  than  $10  million  and  (2)  contracts  valued 
between  $500,000  and  $10  million.  The  higher  dollar  stratum  represented 
the  greatest  portion  of  contract  dollars  awarded  in  fiscal  year  201 2  within 
each  component — ranging  between  79  percent  and  98  percent  of  dollars, 
while  the  lower  dollar  stratum  represented  a  larger  portion  of  contract 
actions  awarded  in  fiscal  year  2012.  Within  each  stratum,  we  selected  a 
nongeneralizable  sample  of  contracts  to  obtain,  to  the  extent  possible,  a 
mix  of  products  and  services  and  a  mix  of  competed  and  noncompeted 
contracts  within  each  component.  We  selected  two  contracts  from  the  first 
stratum  and  three  from  the  second  stratum  (five  per  component)  from  the 
randomized  lists  for  each  component.  Overall,  we  selected  30  contracts 
for  a  more  in-depth  review.  We  excluded  2  of  the  30  selected  contracts 
from  our  final  analysis — 1  from  FHWA  and  one  from  FAA — because,  as  of 
July  31 , 2014,  we  were  unable  to  assess  the  market  research  for  those 
contracts  due  to  missing  market  research  documentation  and  staff 
changes  since  the  award  of  the  contracts.^  As  a  result,  our  analysis  in  this 
report  is  based  on  28  selected  contracts.  For  each  contract,  we  reviewed 


"^For  the  purposes  of  this  report,  we  use  the  term  agency  policy  and  guidance  to  refer  to 
that  issued  by  DOD,  DHS,  DOT,  as  well  as  FAA.  The  FAR  does  not  apply  to  FAA 
acquisitions  pursuant  to  the  Department  of  Transportation  and  Related  Agencies 
Appropriations  Act  of  1996,  Pub.  L.  No.  104-50  §  348  (1995). 

®At  the  conclusion  of  our  review  in  August  2014,  FHWA  officials  reported  they  had  located 
the  preaward  documents  for  the  contract.  For  the  FAA  contract,  contracting  staff  were 
unable  to  locate  the  market  research  for  the  contract.  In  both  instances  the  staff 
responsible  for  conducting,  documenting,  or  accepting  the  market  research  were  no 
longer  with  the  organization. 
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contract  documents  related  to  market  research,  including  market  research 
reports,  acquisition  plans,  justifications  and  approvals  for  sole-source 
awards,  price  negotiation  memorandums  and  independent  government 
cost  estimates,  and  interviewed  or  obtained  information  from  cognizant 
contracting  and  program  officials.  The  findings  from  our  selected 
contracts  are  not  generalizable  to  the  population  of  all  contracts  within  the 
selected  agencies  or  components. 

To  assess  the  reliability  of  the  FPDS-NG  data  used  to  select  the  contracts 
for  review,  we  compared  key  data  elements  from  FPDS-NG  to  the 
contract  documentation,  including  the  contract  number,  agency  name, 
and  whether  or  not  the  contract  had  been  competed  using  full  and  open 
competition.  Where  we  found  discrepancies  relevant  to  our  methodology, 
we  made  adjustments  to  maintain  the  integrity  of  our  selection  criteria. 

For  example,  we  excluded  7  contracts  because  the  contracts  were 
miscoded  in  FPDS-NG  and  therefore  should  have  been  excluded  from 
our  sample.  Specifically,  we  excluded  5  contracts  because  they  were 
small  business  set  asides,  1  contract  because  the  total  dollar  value  fell 
below  the  $500,000  threshold  for  inclusion  in  the  review,  and  1  contract 
because  it  was  a  research  and  development  contract.  Because  we  had 
randomized  lists  of  contracts  for  each  of  the  components,  we  replaced  the 
miscoded  contracts  with  another  contract.  Once  we  corrected  for  these 
errors,  we  concluded  the  data  obtained  was  sufficiently  reliable  for  the 
purposes  of  this  report. 

Market  research  serves  several  purposes,  including  identifying  sources 
capable  of  meeting  the  government’s  need,  promoting  competition, 
identifying  whether  commercial  items  can  meet  the  government’s  needs, 
and  identifying  whether  small  businesses  are  capable  of  meeting  the 
requirement.  For  the  purposes  of  our  review,  we  focused  our  work  on  how 
market  research  informed  the  competition  strategy  and  the  contracting 
officer’s  analysis  of  price  reasonableness.  We  did  not  assess  how  market 
research  informed  the  small  business  determination  or  how  it  informed 
the  decision  to  use  a  commercial  item  given  GAO  has  prior  work  that 
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focuses  on  these  issues.®  Appendix  I  provides  additional  details  on  our 
scope  and  methodology. 

We  conducted  this  performance  audit  from  August  201 3  to  October  2014 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 

Background 

Market  research  has  been  a  statutory  requirement  since  the  passage  of 
the  Competition  in  Contracting  Act  of  1984,  which  required  the  use  of 
market  research  and  acquisition  planning,  and  solicitation  of  proposals,  to 
obtain  competition.^  The  FAR  defines  market  research  as  the  process 
used  to  collect  and  analyze  data  about  capabilities  in  the  market  that 
could  satisfy  an  agency’s  procurement  needs.  The  FAR  provides  general 
policies  and  procedures  for  conducting  market  research  with  the  goal  of 
arriving  at  the  most  suitable  approach  to  acquiring,  distributing,  and 
supporting  supplies  and  services.® 

Market  research  is  conducted  throughout  various  phases  of  an 
acquisition,  and  involves  the  program  office  and  the  contracting  office. 

The  program  office  typically  starts  the  market  research  in  the  acquisition 
planning  phase  once  a  need  is  identified.  The  program  office  may  then 
contact  its  contracting  office  to  coordinate  the  development  of  key 
acquisition  documents.  The  program  office  is  primarily  responsible  for 
conducting  market  research  in  the  early  phases  of  an  acquisition  and 
submitting  a  formal  request  for  contract  to  the  contracting  office. 
Responsibility  for  market  research  shifts  from  the  program  office  to  the 

®For  GAO’S  recent  work  on  commercial  item  determinations,  see  GAO,  Federal 
Contracting:  Commercial  Item  Test  Program  Beneficial,  but  Actions  Needed  to  Mitigate 
Potential  Risks,  GAO-14-178  (Washington  D.C.:  Feb.  4,  2014).  For  GAO’s  recent  work  on 
small  business  contracting,  see  GAO,  Small  Business  Contracting:  Updated  Guidance 
and  Reporting  Needed  for  Consolidated  Contracts,  GAO-14-36  (Washington  D.C.:  Nov. 

26,  2013). 

^Pub.  L  No.  98-369,  Div.  B,  tit.  VII,  §2711. 

®FAR§  10.000. 
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contracting  office  as  the  acquisition  process  progresses  (see  figure  1). 
Further,  the  FAR  requires  that  market  research  be  conducted,  appropriate 
to  the  circumstances,  before  developing  new  requirements  documents.  In 
addition,  the  FAR  identifies  a  number  of  other  points  in  the  acquisition 
process  when  market  research  must  be  conducted,  including  prior  to 
soliciting  offers  for  contracts  above  the  simplified  acquisition  threshold 
and  below  this  threshold  if  adequate  information  is  not  available  and 
circumstances  justify  its  cost.® 


Figure  1:  Market  Research  and  the  Acquisition  Process 


Acquisition  pianning  and  contracting  milestones 

Acquisition  planning  phase 


f Need  identified 

r — 


(procurement 

I  request  approved 


Pstion  ( Offers  received  (contract  awarded 

ted  I 

— V — r — 


Presolicitation 


Program  office  ■ 


Program  and  contracting  office  • 


Preaward 

Postaward 

1 - Contracting  office - 1 

Market  research  is 
conducted  before 
developing  new 
requirements 
documents. 


Market  research 

Market  research  conducted 


Market  research  is  conducted 
before  soliciting  offers  for: 

•  Acquisitions  above  the 
simplified  acquisition  threshold;^ 

•  Acquisitions  below  the  simplified 
acquisition  threshold,  in  certain 
circumstances: 

•  Acquisitions  that  could  lead  to  a 
bundled  contract.^ 


Market  research  can  be 
used  to  inform  price 
reasonableness  during 
the  evaluation  of 
proposals. 


Market  research  is 
conducted  before 
awarding  task  or 
delivery  orders  under 
an  existing  contract  in 
certain  circumstances. 


Source;  GAO  analysis  of  federal  regulations  and  agency  guidance.  |  GAO-15-8 

®The  simplified  acquisition  threshoid  is  generaiiy  $150,000. 

‘’Bundiing  refers  to  the  consoiidating  of  two  or  more  requirements  for  goods  or  services  previousiy 
provided  or  performed  under  separate  smaiier  contracts  into  a  soiicitation  for  a  singie  contract  that  is 
iikeiy  to  be  unsuitabie  for  award  to  a  smaii  business  concern. 


®FAR  §  10.001(a)(2).  The  simplified  acquisition  threshold  is  generally  $150,000. 
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Federal  regulations  provide  some  flexibility  in  terms  of  how  and  to  what 
extent  market  research  should  be  conducted,  considering  the  nature  of 
the  procurement.  Specifically,  the  FAR  provides  that  the  extent  of  market 
research  depends  on  a  number  of  factors,  such  as  urgency,  estimated 
dollar  value,  complexity,  and  past  experience. The  FAR  also  states  that 
agencies  should  document  the  results  of  market  research  in  a  manner 
appropriate  to  the  size  and  complexity  of  the  acquisition,  but  does  not 
prescribe  any  particular  format  or  specific  elements  for  documentation. 

We  have  issued  a  number  of  reports  with  implications  for  market 
research.  In  our  201 1  report  on  acquisition  planning,  we  note  that  market 
research  is  a  part  of  the  acquisition  planning  process,  a  process  that  is 
responsible  for  successful  acquisition  outcomes. In  addition,  we  issued 
several  reports  related  to  the  market  research  objective  of  promoting 
competition.  For  example,  in  a  January  2012  report  reviewing  DOD’s  use 
of  the  national  security  exception  (which  allows  agencies  to  limit 
competition  for  a  contract  in  instances  when  disclosure  of  an  agency’s 
needs  would  compromise  national  security),  as  well  as  in  a  March  2013 
report  reviewing  DOD’s  use  of  competitive  procedures,  we  found  few 
details  about  the  market  research  findings  that  helped  justify  the  decision 
to  pursue  a  noncompetitive  or  limited  competition  award  for  some 
contracts  reviewed.''^  In  contrast,  our  best  practices  work  has  shown  how 
market  research  can  be  used  to  enhance  competition.  Specifically,  in  an 
April  2013  report  identifying  leading  commercial  company  practices  for 
acquiring  services,  we  reported  that  one  company  sometimes  chose  to 
execute  a  short  term  contract  to  buy  time  if  market  research  showed  a 
more  competitive  deal  could  be  obtained  at  a  later  date.''^ 


^°FAR§  10.002(b)(1). 

^^GAO,  Acquisition  Pianning:  Opportunities  to  Buitd  Strong  Foundations  for  Better 
Services  Contracts,  GAO-11-672  (Washington,  D.C.:  Aug.  9,  2011). 

^^GAO,  Defense  Contracting:  improved  Poiicies  and  Toois  Couid  Heip  increase 
Competition  on  DOD’s  Nationai  Security  Exception  Procurements,  GAO-12-263 
(Washington,  D.C.:  Jan.  13,  2012);  GAO,  Defense  Contracting:  Actions  Needed  to 
increase  Competition,  GAO-13-325  (Washington,  D.C.:  Mar.  28,  2013). 

^^GAO,  Strategic  Sourcing:  Leading  Commerciai  Practices  Can  Heip  Federai  Agencies 
increase  Savings  When  Acquiring  Services,  GAO-13-417  (Washington,  D.C.:  Apr.  15, 
2013). 
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While  market  research  is  one  factor  that  may  affect  competition,  there  are 
a  number  of  other  factors  as  well,  such  as  whether  the  government  owns 
the  rights  to  the  technical  data  on  a  contract  and  the  relationship  between 
a  program  office  and  a  contracting  office.  For  example,  in  a  May  2014 
report  reviewing  DOD’s  noncompetitive  and  one-offer  awards,  we  found 
that  the  justifications  for  half  of  the  noncompetitive  contracts  and  task 
orders  stated  that  the  awards  could  not  be  competed  due  to  a  lack  of 
technical  data.'''^  Similarly,  in  a  March  2014  report  reviewing  the  use  of  the 
unusual  and  compelling  urgency  exception  to  full  and  open  competition  at 
three  federal  agencies,  we  found  that  for  10  DOD  contracts  reviewed,  the 
government  was  unable  to  compete  requirements  in  part  because  of  a 
lack  of  access  to  technical  data  packages  or  proprietary  data.''®  In  terms 
of  the  relationship  between  the  contracting  and  program  offices,  in  July 
2010  we  reported  that  several  agencies  recognized  that  program  offices 
place  pressure  on  the  contracting  process  to  award  new  contracts  to  a 
specific  vendor  without  competition.  Further,  we  reported  that  many 
contracting  officials  we  spoke  with  recognized  that  program  staff 
sometimes  prefer  a  specific  vendor — often  an  incumbent — in  some  cases 
because  a  relationship  has  developed  between  the  program  office  and 
the  contractor,  who  understands  the  program  requirements.''®  We  also 
found  that  a  strong  incumbent,  sometimes  coupled  with  overly  restrictive 
government  requirements,  are  factors  that  may  contribute  to  receiving 
only  one  offer  on  a  competitive  procurement. 


1  d 

GAO,  Defense  Contracting:  Early  Attention  in  the  Acquisition  Process  Needed  to 
Enhance  Competition,  GAO-14-395  (Washington,  D.C.,  May  5,  2014). 

^®GAO,  Federal  Contracting:  Noncompetitive  Contracts  Based  on  Urgency  Need 
Additional  Oversight,  GAO-14-304  (Washington,  D.C.:  Mar.  26,  2014). 

^®GAO,  Federal  Contracting:  Opportunities  Exist  to  Increase  Competition  and  Assess 
Reasons  When  Only  One  Offer  is  Received,  GAO-1 0-833  (Washington,  D.C.:  Jul.  26, 
2010). 
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Selected  Agencies’ 
Market  Research 
Guidance  Is 
Generally  Consistent 
with  Federal 
Regulations,  but 
Documentation 
Requirements  Vary  at 
Some  Agencies 


We  found  that  agencies’  market  research  guidance  at  DOD,  DHS,  DOT, 
and  FAA  is  consistent  with  federal  regulations  in  terms  of  market  research 
objectives,  techniques,  and  documentation,  but  agencies’  guidance  varies 
from  one  another  on  the  specificity  of  documentation  required.  For 
example,  DHS  guidance  requires  that  specific  elements  of  market 
research  be  documented  and  DOD  and  FAA  make  recommendations  on 
the  types  of  elements  to  be  documented.  In  contrast,  DOT  provides 
acquisition  staff  limited  instruction,  stating  that  the  level  of  detail  might 
vary  based  on  factors  such  as  whether  the  market  research  is  for  a  one¬ 
time  or  recurring  requirement  but  its  market  research  guidance  does  not 
require  any  specific  elements  to  be  documented.  Based  on  our  review  of 
agencies’  market  research  guidance,  we  identified  four  basic  market 
research  elements,  which,  if  recorded,  would  allow  those  not  connected 
with  the  research  to  understand  how  the  acquisition  team  collected  and 
analyzed  information  about  capabilities  within  the  market  to  satisfy 
agency  needs.  These  four  market  research  elements  include  (1)  the 
methods  used,  (2)  the  timeframes  when  staff  used  them,  (3)  an  analysis 
of  the  capabilities  of  potential  sources,  and  (4)  a  conclusion  based  on  this 
analysis. 


Selected  Agencies’ 
Guidance  Is  Consistent 
with  FAR  Objectives  for 
Market  Research  and 
Provides  Additional 
Techniques  for 
Communicating  with  the 
Industry 


Our  analysis  found  that,  consistent  with  the  FAR,  selected  agencies’ 
guidance  indicates  that  market  research  may  be  used  to  inform  two  key 
decisions.  The  first  decision — deciding  whether  or  not  to  conduct  a  full 
and  open  competition — is  made  as  part  of  the  acquisition  planning 
process  and  one  of  the  purposes  of  market  research  is  to  promote 
competition  to  the  maximum  extent  practicable.''^  Federal  regulations 
require  agencies  to  perform  acquisition  planning  activities  for  all 
acquisitions  to  ensure  that  the  government  meets  its  needs  in  the  most 
effective,  economical,  and  timely  manner  possible.''®  The  second 
decision — deciding  whether  the  prices  to  be  paid  for  the  goods  or 
services  is  reasonable — is  made  prior  to  the  award  of  a  contract.  This 
decision  is  commonly  referred  to  as  the  “price  reasonableness”  decision. 
While  not  required,  the  FAR  identifies  market  research  as  a  method  for 
this  purpose.'®  More  specific  information  about  the  FAR  provisions  and 
how  agency  guidance  reinforces  them  follows: 


'Tar  §  7.102(a). 

'®FAR§  7.102(a)  and  (b). 
'®FAR§  15.404-1  (b)(2)(vi). 
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•  Competition:  The  FAR  requires,  among  other  things,  that  market 
research  and  acquisition  planning  be  used  to  promote  and  provide  for 
full  and  open  competition  or,  if  not  required,  to  obtain  competition  to 
the  maximum  extent  practicable. Agencies’  guidance  reinforces  the 
goal  of  providing  for  competition  in  several  ways,  for  example,  by 
instructing  acquisition  staff  to  identify  and  analyze  the  capabilities  of 
potential  sources  prior  to  making  the  competition  decision,  which 
would  include  determining  whether  the  requirement  should  be  set 
aside  for  small  business  concerns,  open  to  full  and  open  competition, 
or  sole-sourced  to  a  specific  vendor.^''  DOD  and  DOT  guidance 
explains  that  market  research  helps  the  acquisition  team  become 
informed  consumers,  enabling  the  team  to  develop  the  optimal 
procurement  strategy.  DHS  guidance  notes  that  market  research 
should  be  used  to  refine  requirements  to  maximize  the  benefit  of 
competitive  market  forces,  as  needed,  and  to  consider  if  modifying  a 
requirement  may  allow  a  commercial,  rather  than  a  noncommercial, 
item  to  be  acquired. 

•  Price  Reasonableness:  The  FAR  identifies  the  comparison  of 
proposed  prices  on  a  contract  with  prices  obtained  through  market 
research  for  the  same  or  similar  items  as  one  of  several  methods  for 
determining  price  reasonableness.^^  Agencies’  guidance  confirms  the 
usefulness  of  market  research  in  informing  what  price  would  be 
reasonable  for  the  government  to  pay  for  a  procurement.  For 
example,  FAA  guidance  states  that  market  research  and  analysis  can 
help  ensure  that  the  government’s  needs  are  met  in  a  cost  effective 
manner  based  on  understanding  prices  in  the  commercial 
marketplace.  DOD  guidance  similarly  explains  that  knowledge  of 
market  pricing  information  is  necessary  for  planning  for  contract 
obligations  and  determining  price  reasonableness. 

Agencies’  guidance  builds  upon  the  market  research  techniques  identified 
in  the  FAR  by  identifying  a  greater  variety  of  ways  of  communicating  with 
industry  and  emphasizing  the  importance  of  doing  so.^^  The  market 


^°FAR  §  7.102(a)(2).  The  FAR  also  directs  that  the  results  of  market  research  are  to  be 
used  to  determine  if  sources  capable  of  satisfying  the  agency’s  requirements  exist.  FAR  § 
1 0.001  (a)(3)(i). 

^^Subpart  6.3  of  the  FAR  specifies  the  conditions  under  which  an  agency  may  award  a 
contract  to  a  single  source  without  providing  for  full  and  open  competition.  The  award  of 
such  a  contract  can  be  referred  to  as  a  sole-source  award. 

22fAR§  15.404-1  (b)(2)(vi). 

oo 

Section  10.002(b)(2)  of  the  FAR  lists  techniques  for  conducting  market  research. 
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research  techniques  described  in  the  FAR,  as  well  as  those  in  agencies’ 
guidance,  can  be  broadly  categorized  as  those  used  by  acquisition  staff 
to  (1)  obtain  information  from  internal  sources,  such  as  reviewing  the 
results  of  recent  market  research  undertaken  to  meet  similar  or  identical 
requirements,  or  (2)  communicate  directly  with  external  sources, 
particularly  industry  sources  (see  table  1). 


Table  1:  Market  Research  Techniques  Identified  Under  Part  10  of  the  Federal  Acquisition  Regulation  (FAR)  and  in  Agency 
Guidance 

FAR 

DOD 

DHS 

DOT  FAA 

Internal  Information 

Reviewing  the  results  of  recent  market  research  undertaken  to  meet  similar  or  identical 
requirements 

Contacting  knowledgeable  government  officials  on  market  capabilities  to  meet  requirements, 
including  obtaining  source  lists  of  similar  items  from  other  contracting  activities  or  agencies 

✓ 

■/ 

Querying  the  government-wide  database  of  contracts  and  other  procurement  instruments 
intended  for  use  by  multiple  agencies  available  at  www.contractdirectory.gov/contractdirectory/ 
and  other  government  and  commercial  databases 

■/ 

Reviewing  catalogs  and  other  generally  available  product  literature  published  by 
manufacturers,  distributors,  and  dealers  or  available  on-line 

■/ 

Reviewing  prior  acquisition  history 

External  Information 

Contacting  knowledgeable  industry  officials  on  market  capabilities  to  meet  requirements  or 
obtaining  source  lists  of  similar  items  from  trade  associations  or  other  sources 

Conducting  interchange  meetings  or  holding  presolicitation  conferences  to  involve  potential 
offerors  early  in  the  acquisition  process 

✓ 

,/ 

Publishing  formal  requests  for  information  in  appropriate  technical  or  scientific  journals  or 
business  publications 

■/ 

y 

Participating  in  interactive,  on-line  communication  among  industry,  acquisition  personnel,  and 
customers 

■/ 

V 

Holding  one-on-one  meetings  with  industry 

•/ 

✓ 

Publishing  requests  for  information  or  sources  sought  notices  at  www.FedBizOpps.gov® 

■/ 

Attending  trade  fairs  and  conferences 

■/ 

■/ 

Performing  site  visits  (program  staff  visiting  vendors  or  vendors  visiting  the  agency) 

•/ 

■/ 

Preparing  a  formal  survey  or  questionnaire  and  sending  it  to  multiple  suppliers 

✓ 

■/ 

Sponsoring  industry  conferences  or  industry  days 

✓ 

■/ 

Source:  GAO  analysis  of  part  1 0  of  the  FAR  and  of  agency  guidance.  |  GAO-1 5-8 


“Federal  Business  Opportunities,  commonly  referred  to  as  FedBizOpps,  is  a  web-based  government 
portal  which  allows  all  potential  vendors  to  consider  federal  procurement  opportunities. 
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Agencies’  guidance  generally  mirrors  the  FAR  in  terms  of  potential 
internal  techniques  that  could  be  used  to  conduct  market  research  and 
provides  additional  means  to  communicate  with  industry  that  are  not 
specified  in  the  FAR.  For  example,  while  the  FAR  suggests  that 
acquisition  staff  place  a  request  for  information  in  technical  or  scientific 
journals  or  business  publications,  DOD,  DHS,  and  DOT  guidance 
expands  upon  this  suggestion  by  indicating  that  requests  for  information 
could  be  posted  on  FedBizOpps.  A  contracting  officer  could  use  such  a 
posting  to  solicit  industry  response  to  a  questionnaire  or  obtain  industry 
comment  on  a  draft  statement  of  work  on  behalf  of  an  acquisition  team. 
Further,  a  contracting  officer  can  post  a  sources  sought  notice — a  notice 
requesting  that  vendors  interested  in  providing  a  particular  product  or 
service  identify  themselves — on  FedBizOpps.  DHS  guidance  notes  that 
vendors  who  respond  to  the  notice  usually  describe  their  capabilities  or 
similar  projects  they  have  conducted,  among  other  things.  While  DOD 
guidance  also  recommends  these  techniques,  it  also  suggests  that 
acquisition  staff  take  additional  measures  to  reach  vendors  that  have  not 
previously  bid  on  government  contracts,  such  as  advertising  in  business 
publications.  In  addition,  DOD  guidance  recommends  attending  trade 
shows  and  conferences  as  a  good  source  of  general  information  about  an 
industry  and  DHS  guidance  identifies  these  events  as  places  to  meet 
industry  leaders  and  vendors. 

To  promote  industry  outreach,  each  agency  in  our  review  has  developed 
guidance,  known  as  vendor  communication  plans,  designed  to  facilitate 
interaction  between  acquisition  staff  and  vendors.  Agencies’  actions  were 
consistent  with  an  OMB  initiative  intended  to  encourage  acquisition  staff 
to  have  productive  interactions  with  industry  partners  during  the 
acquisition  process. In  a  February  2,  201 1  memorandum,  OMB  stated 
that  such  outreach  is  important  so  that  the  government  has  a  clear 
understanding  of  the  marketplace  and  can  award  a  contract  for  an 
effective  solution  at  a  reasonable  price. The  memorandum  further  states 
that  acquisition  staff  were  not  taking  full  advantage  of  flexibilities  offered 
in  the  FAR  because  of  their  concern  that  vendors  could  misinterpret  the 
discussions  as  obligating  the  government  to  conduct  business  with  them 
and  that  agencies  must  remove  unnecessary  barriers  to  reasonable 


^“^FAA  published  its  vendor  communication  plan  in  July  2009,  prior  to  the  OMB  initiative. 

^^See  OMB,  “Myth-Busting”  Memorandum  (Washington,  D.C.:  Feb.  2,  2011)  and  OMB, 
“Myth-Busting  2”  Memorandum  (Washington,  D.C.:  May  7,  2012). 
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communication  with  industry.  DOD  and  DHS  guidance  cite  the 
importance  of  clear  and  frequent  communication  between  government 
and  industry  because  of  benefits  for  improved  acquisition  outcomes, 
including  opening  the  door  for  more  small  business  participation. 

While  agencies’  guidance  identifies  a  broad  array  of  market  research 
techniques,  the  guidance  also  notes  that  the  extent  or  type  of  market 
research  will  vary,  depending  on  factors  including  urgency,  estimated 
dollar  value,  complexity,  and  past  experience.  For  example,  DHS 
guidance  notes  that  several  methods  of  obtaining  information  from 
industry,  such  as  sponsoring  presolicitation  conferences  to  discuss 
industry  comments  on  proposed  solicitations,  are  conducted  more 
typically  for  complex  requirements.  DOD  guidance  recognizes  the 
importance  of  market  research  on  lower  dollar  procurements,  noting  that 
DOD  awards  a  multitude  of  lower-dollar  contracts  for  new  products  or 
services  and  that  the  goals  of  market  research  to  understand  the 
commercial  market  remain  the  same  for  these  procurements.  DOD  and 
DOT  guidance  suggests  that  acquisition  staff  weigh  the  administrative 
cost  of  conducting  market  research  to  the  potential  benefits.  Similarly, 
DHS  guidance  suggests  that  the  acquisition  team  solicit  the  contracting 
officer’s  advice  to  avoid  requesting  more  than  the  minimum  information 
necessary  from  potential  sources  when  conducting  market  research.  FAA 
guidance  suggests  that  market  research  may  range  from  a  telephone  call 
to  formal  market  surveys  or  solicitations  requesting  information. 


Selected  Agencies 
Require  Market  Research 
Be  Documented,  but  Vary 
in  the  Amount  of  Specific 
Information  Required 


All  four  agencies  require  that  market  research  be  clearly  documented 
since  it  can  inform  current  and  future  procurements,  but  the  guidance 
varies  in  terms  of  the  elements  that  are  to  be  included  in  the 
documentation.  The  FAR  requires  that  for  more  complex  and  higher  dollar 
procurements,  acquisition  personnel  prepare  written  acquisition  plans  that 
include  a  description  of  prospective  sources  and  address  the  extent  and 
results  of  market  research  and  the  impact  of  these  findings  on  various 
elements  of  the  plan.^®  Consistent  with  the  FAR,  all  of  the  agencies  in  our 
review  have  guidance  regarding  acquisition  planning  for  procurements 
designated  as  more  complex  or  higher  value.  At  DOD,  DHS,  and  DOT, 
written  acquisition  plans  are  typically  required  for  higher  valued 


^®FAR  §  7.105(b)(1).  FAR  §  7.103(e)  directs  that  agency  heads  determine  criteria  and 
thresholds  under  which  increasingly  greater  detail  and  formality  in  the  planning  process  is 
required. 
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acquisitions.  For  example,  at  DHS,  these  plans  are  typically  required  for 
acquisitions  of  $10  million  and  above  and,  at  DOT,  written  acquisition 
plans  are  generally  required  for  procurements  greater  than  $20  million. 
FAA  has  similar  requirements  for  acquisition  plans  for  most  procurements 
greater  than  $25,000.  In  addition,  in  most  cases  where  a  product  or 
service  is  to  be  acquired  on  a  sole-source  basis,  the  FAR  requires  that 
contracting  officers  prepare  a  written  justification,  also  known  as  a 
justification  for  other  than  full  and  open  competition,  which  is  to  include  a 
description  of  the  market  research  conducted  or  the  reason  that  market 
research  was  not  conducted  as  a  part  of  justifying  that  a  contract  be  sole- 
sourced.^^  The  agencies  in  our  review  all  have  guidance  regarding 
justifications  for  sole-source  contracts,  and  DHS  and  FAA’s  guidance 
specifically  address  the  requirement  to  describe  the  market  research  in 
support  of  the  sole-source  award. 

Beyond  the  market  research  information  required  as  part  of  acquisition 
plans  for  more  complex,  higher  dollar  value  procurements,  or  as  part  of 
the  justification  and  approval  for  sole-source  procurements,  we  found  that 
the  specific  type  of  market  research  information  that  must  be  documented 
for  all  procurements,  including  contracts  of  lower  value,  varies 
considerably  by  agency.  Only  DHS  required,  as  of  January  2014,  that 
specific  elements — such  as  who  conducted  the  market  research,  the 
methods  used,  and  the  outcome  and  impact  of  the  market  research — be 
documented.  In  contrast,  DOD,  DOT,  and  FAA  do  not  require  that  specific 
elements  be  recorded  consistently,  although  DOD  and  FAA  guidance 
contain  recommendations  on  elements  that  could  be  recorded  (see  figure 
2). 


^^FAR  §  6.303-2(b)(8).  The  FAR  outlines  similar  requirements  for  orders  and  blanket 
purchase  agreements  above  the  simplified  acquisition  threshold  established  under  the 
federal  supply  schedules  without  using  competitive  procedures.  See  FAR  §  8.405- 
6(c)(2)(vi). 
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Figure  2:  Required  and  Recommended  Market  Research  Eiements  as  of  Juiy  2014 


Agency  Specific  eiements  are  Specific  eiements  are 

required  by  poiicy  recommended  in 

guidance  ortempiate 

Department  of 
Defense 

Department  of 
Homeland  Security 

Department  of 
Transportation 

\\ 

Federal  Aviation 
Administration 

\  \ 

Required  Elements  at  DHS 

Department  of  Homeland  Security  Acquisition  Manual: 

A  market  research  report  is  required  for  contract  actions  awarded 
above  the  simplified  acquisition  threshold  and  shall  provide,  at  a 
minimum,  the  following  information:® 

1.  Participants  in  the  market  research  effort  and  responsibilities  of 
team  members; 

2.  Market  research  methods  used;  and 

3.  Outcome  of  the  market  research  identifying  potential  sources 
and  findings  that  impact  the  procurement. 

Source:  GAO  analysis  of  DOD,  DHS,  DOT,  and  FAA  guidance.  [  GAO-15-8 

®The  simplified  acquisition  threshold  is  generally  $150,000. 


DOD  and  FAA  market  research  guidance  recommends,  but  does  not 
require,  documenting  elements  similar  to  the  ones  DHS  requires  be 
documented.  In  addition,  DOD  guidance  recommends  documenting  the 
timeframes  during  which  the  market  research  was  conducted,  a 
description  of  commercial  business  practices,  and  related  cost  and  price 
factors.  DOD,  DHS,  and  FAA  provide  templates  or  sample  memorandums 
for  the  contract  file  that  contain  placeholders  for  recommended 
information.^®  DOT  market  research  guidance  does  not  specify  elements 
to  be  documented,  and  notes  that  the  type  and  amount  of  information  to 
be  kept  should  be  based  on  how  it  may  be  used  in  the  future  and  whether 
the  documentation  is  for  a  one-time,  unique,  or  recurring  requirement. 

DOD,  DHS,  and  FAA  have  taken  steps  to  enhance  the  documentation  of 
market  research  information  over  the  past  few  years.  For  example: 


oo 

DOD’s  template  applies  specifically  to  services  acquisitions. 
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•  In  April  2012,  DOD  officials  issued  a  template,  including 
recommended  information,  for  preparing  market  research  reports  for 
services  acquisitions.  The  template  was  prepared  by  a  working  group 
that  identified  best  practices  for  conducting  and  documenting  market 
research  to  allow  for  more  effective  collection  and  sharing  of  market 
research  across  DOD.  The  first  part  of  the  template  is  for  recording 
basic  information,  such  as  a  description  of  vendor  capabilities  and 
small  business  opportunities.  The  second  part  of  the  template  is 
targeted  toward  more  complex  high-dollar  value  procurements  and 
requires  more  detailed  information  about  the  commercial  market,  such 
as  a  description  of  the  demand  for  the  service  and  an  assessment  of 
fair  and  reasonable  prices.  DOD  officials  reported  to  us  that  they  are 
considering  whether  use  of  the  template  should  be  mandatory. 

•  In  October  2012,  FAA  issued  its  sample  Market  Research 
Memorandum  to  the  File  with  recommended  elements  to  record,  and 
in  April  2013,  FAA  first  required  the  use  of  written  acquisition  plans. 
The  acquisition  plan  template  for  acquisitions  above  the  simplified 
acquisition  threshold  instructs  that  the  results  of  market  research  be 
recorded,  as  well  as  the  size  and  status  of  each  source,  and  a 
competition  determination — for  example,  whether  the  work  should  be 
set  aside  for  small  businesses. 

•  DHS  had  previously  included  in  guidance  that  minimum  elements  be 
documented.  In  January  2014  DHS  incorporated  these  required 
elements  into  the  Department  of  Homeland  Security  Acquisition 
Manual.  According  to  DHS  officials,  by  doing  so,  the  agency  elevated 
the  importance  of  including  these  elements  in  market  research 
reports. 

Based  on  our  analysis  of  market  research  elements  identified  in  DOD, 
DHS,  and  FAA  policy,  guidance,  or  templates,  we  identified  and 
discussed  with  agencies’  officials  four  basic  market  research  elements, 
which,  if  recorded,  would  allow  those  not  connected  with  the  research  to 
understand  how  the  acquisition  team  collected  and  analyzed  information 
about  capabilities  within  the  market  to  satisfy  agency  needs.  These  four 
market  research  elements  include  (1)  the  methods  used,  (2)  the 
timeframes  when  staff  used  them,  (3)  an  analysis  of  the  capabilities  of 
potential  sources,  and  (4)  a  conclusion  based  on  this  analysis.  DOD, 

DHS,  and  FAA  market  research  policies,  guidance,  or  templates  included 
at  least  three  of  the  four  basic  elements.  For  example,  at  DHS,  three  of 
these  elements  were  included  in  the  Department  of  Homeland  Security 
Acquisition  Manual  and  the  fourth  element,  timeframes,  was  included  in 
DHS’s  market  research  template.  However,  DOT’S  market  research 
guidance  did  not  include  any  specific  elements.  Internal  control  standards 
state  that  control  activities,  such  as  policies  and  procedures,  help  ensure 
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that  management  directives  are  carried  out  and  actions  are  taken  to 
address  risks. Requiring  that  specific  market  research  elements  be 
documented  in  contract  files  would  help  ensure  that  documentation  is 
readily  available  to  inform  current  and  future  procurements. 


Selected  Agencies 
May  Have  Missed 
Opportunities  to 
Promote  Competition 
and  Inform  Future 
Procurements  on 
Lower  Dollar 
Contracts 


We  found  that  all  28  contracts  we  reviewed  included  some  evidence  of 
market  research,  but  the  market  research  on  higher  dollar  value 
contracts — those  valued  at  $10  million  or  more — tended  to  be  more 
robust  and  appeared  to  help  promote  competition.  Agencies  did  not  take 
advantage  of  many  available  market  research  techniques  on  lower  dollar 
value  contracts,  and  as  a  result  may  have  missed  opportunities  to 
promote  competition.  In  10  of  the  12  higher  dollar  contracts  we  reviewed, 
agency  officials  conducted  outreach  to  vendors  through  issuance  of 
requests  for  information,  draft  requests  for  proposals,  or  other  techniques. 
In  some  cases,  this  approach  resulted  in  the  agency  modifying  or 
clarifying  its  requirements  to  encourage  competition,  and  all  but  one  of 
the  higher  dollar  contracts  that  conducted  some  type  of  formal  outreach  to 
industry  received  more  than  one  offer.  In  contrast,  in  only  4  of  the  16 
lower  dollar  contracts  did  DOD,  DHS,  and  DOT  officials  conduct  formal 
outreach  to  industry,  instead  often  relying  on  issuance  of  a  notice  that  the 
agency  intended  to  award  a  contract  on  a  sole-source  basis  or  on  a 
review  of  the  government’s  acquisition  history.  In  several  cases,  DOD  and 
DHS  officials  relied  on  market  research  that  appeared  to  be  incomplete  or 
outdated.  Overall,  we  identified  limitations  in  the  market  research 
supporting  seven  lower  dollar  value  contracts,  all  of  which  were  sole- 
sourced  or  received  only  one  offer.  Similarly,  while  all  28  contracts  we 
reviewed  contained  evidence  of  market  research,  agency  officials  often 
did  not  document  one  or  more  of  the  four  basic  market  research  elements 
which  we  identified  based  on  our  review  of  agencies’  guidance,  potentially 
limiting  the  ability  to  use  this  market  research  to  inform  future 
procurements.  We  also  found  that  agencies’  officials  used  market 
research  to  inform  price  reasonableness  determinations  in  most  of  the 
contracts  we  reviewed,  though  other  techniques  were  also  frequently 
used  to  inform  pricing  determinations. 


OQ 

GAO,  Internal  Control:  Standards  for  Internal  Control  In  the  Federal  Government, 
GAO/AIMD-00-21.3.1  (Washington,  D.C.:  November,  1999). 
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Market  Research  on 
Higher  Dollar  Value 
Contracts  Involved  More 
Communication  with 
Industry  and  Contributed 
to  Competition 


We  found  that  agencies’  market  research  on  the  higher  dollar  value 
contracts  generally  involved  more  communication  with  industry  and  these 
contracts  were  more  often  awarded  on  a  competitive  basis  than  were  the 
lower  dollar  value  contracts  we  reviewed  (see  figure  3). 


Figure  3:  Market  Research  Techniques  and  Competition  Resuits  on  28  Contracts 
GAO  Reviewed 

Higher  dollar  (above  $10  million) 


1 2  contracts  reviewed 


Lower  dollar  (between  $500,000  and  $10  million) 
1 6  contracts  reviewed 


O  O 


Degree  of  competition  achieved 

Multiple  offers  received 
One  offer  received 
Not  competed 


Market  research  techniques 

Included  communication  with  industry 


Source:  GAO  analysis  of  DOD,  DHS,  DOT,  and  FAA  documentation.  ]  GAO-15-8 

Note:  For  the  purposes  of  this  analysis,  we  defined  communication  with  industry  to  include  attending 
industry  trade  shows  or  conducting  site  visits,  contacting  multiple  potential  vendors  and  using  their 
feedback  to  update  or  clarify  the  requirement,  or  publishing  formal  requests  for  information,  sources 
sought  notices,  or  draft  solicitations  in  FedBizOpps. 


In  particular,  we  found  that  for  higher  dollar  contracts,  agencies’  officials 
made  more  extensive  use  of  techniques  involving  communication  with 
industry,  such  as  issuing  requests  for  information  and  draft  requests  for 
proposals,  and  holding  conferences  with  vendors — using  these  types  of 
techniques  on  10  of  the  12  higher  dollar  value  contracts  we  reviewed. 
These  methods  allowed  agencies’  officials  to  gather  vendor  feedback  and 
collect  information  that  could  be  used  to  refine  requirements  and  inform 
competition  determinations.  The  lower  dollar  contracts  we  reviewed  relied 
more  extensively  on  internal  sources  of  information,  such  as  the 
knowledge  of  program  officials,  examinations  of  acquisition  history,  or  on 
the  issuance  of  notices  of  intent  to  sole-source  as  the  main  sources  of 
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market  research,  rather  than  on  formal  outreach  to  industry. Specifically, 
while  market  research  on  4  of  the  16  lower  dollar  contracts  we  reviewed 
involved  outreach  to  industry,  there  was  no  outreach  to  industry  on  the 
other  12  contracts  reviewed.  The  FAR  states  that  agencies  are 
encouraged  to  promote  early  exchanges  of  information  about  future 
acquisitions  to,  among  other  things,  improve  the  understanding  of  the 
government’s  requirement  and  industry  capabilities. 

The  following  examples  highlight  some  market  research  techniques  used 
by  agencies’  officials  on  higher  dollar  value  contracts,  conclusions 
reached  based  on  the  market  research,  and  competition  outcomes: 

•  The  FAA  awarded  a  $2  billion  contract  for  data  communication 
integrated  services  after  conducting  14  formal  requests  for  information 
and  other  outreach  to  potential  vendors  over  a  4-year  period.  The 
contracting  staff  used  the  information  gathered  from  industry  to  help 
develop  the  final  requirement.  FAA  received  offers  from  three  vendors 
it  had  identified  as  likely  to  make  offers  based  on  its  market  research 
efforts. 

•  Similarly,  on  an  $87  million  DHS  contract  for  guard  services,  the 
contracting  specialist  identified  thousands  of  potential  contractors 
through  a  database,  narrowed  this  number  to  172  potential  vendors 
by  cross  referencing  the  vendor  information  with  online  information 
about  vendors  with  existing  government-wide  contracts,  issued  a 
sources  sought  notice,  evaluated  26  responses  to  the  notice,  and 
ultimately  concluded  that  there  were  enough  sources  capable  of 
meeting  the  need  to  allow  for  full  and  open  competition.  Eighteen 
vendors  made  offers  on  the  contract. 

•  In  a  $72  million  DHS  acquisition  for  program  management  and 
logistics  support,  a  total  of  31  contractors  responded  to  either  a 
request  for  information  posted  in  2009  and/or  a  subsequent  one 
posted  in  2011.  The  feedback  obtained  through  the  requests  for 
information  led  to  the  conclusion  that  the  acquisition  strategy  should 
be  revised  to  address  industry  questions,  comments,  and  concerns, 
including  issues  with  funding  and  performance  related  to  the  contract 
structure.  Agency  officials  therefore  issued  a  sole-source  bridge 
contract,  in  part  so  that  they  could  revise  the  requirement. 

Additionally,  the  market  research  report  for  this  contract  stated  that 


on 

FAR  Part  5.2  generally  requires  agencies  to  publish  notice  on  FedBizOpps  of  proposed 
contract  actions. 
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while  all  31  contractors  responding  to  the  requests  for  information 
might  not  make  offers  or  be  capable  of  fulfilling  the  requirement,  the 
information  gathered  from  the  responses  gave  the  agency  insight  into 
the  capabilities  of  the  marketplace,  and  led  to  the  conclusion  that 
there  would  be  ample  qualified  vendors  in  the  market  to  meet  the 
requirements  of  a  subsequent  contract.  According  to  an  agency 
official,  the  award  of  the  follow-on  contract  is  now  in  process — being 
competed  among  certain  small  business  set  aside  vendors — and  the 
agency  has  received  multiple  offers.  The  offers  are  currently  being 
evaluated,  and  DHS  officials  anticipate  awarding  the  contract  around 
the  close  of  fiscal  year  2014. 

Overall,  our  work  found  the  market  research,  and  in  particular,  industry 
outreach  efforts,  appeared  to  contribute  to  the  degree  of  competition 
achieved  on  the  higher  dollar  contracts  we  reviewed.  For  example,  9  of 
the  12  higher  dollar  contracts  we  reviewed  received  more  than  one  offer; 
and  in  all  but  one  of  these  cases,  agencies’  officials  conducted  some  type 
of  formal  outreach  efforts  to  industry. 

While  the  higher  dollar  value  contracts  tended  to  be  competed  and  most 
involved  outreach  with  industry,  we  found  additional  market  research 
actions  could  have  potentially  promoted  competition  on  a  3-year,  $210 
million  DLA  contract  for  tax-free  fuel  in  Italy  awarded  in  201 1 .  In  this  case, 
DLA  enables  U.S.  military  personnel  to  purchase  fuel  on  a  tax-free  basis 
while  stationed  there.  To  do  so,  DLA  issued  a  solicitation  that  required 
potential  offerors,  among  other  things,  to  (1)  have  at  least  1,000  service 
stations  in  Italy  and  (2)  be  able  to  accept  printed  coupons  that  allowed 
military  personnel  to  buy  the  fuel  on  a  tax-free  basis.  The  market  research 
performed  by  program  officials  consisted  of  a  list  of  all  of  the  fuel 
companies  operating  in  Italy;  however,  contracting  officials  confirmed  that 
only  two  of  the  nine  companies  on  the  list  were  likely  to  be  able  to  meet 
the  requirement  of  1,000  gas  stations  included  in  the  solicitation.  Both  of 
these  companies  were  incumbents  on  the  prior  contract.  However, 
contracting  officials  stated  that  one  of  the  two  companies,  which  was  an 
incumbent,  did  not  submit  an  offer  because  the  continued  use  of  printed 
coupons  was  too  cumbersome  for  them.  After  receiving  a  single  offer,  the 
contracting  officer  asked  the  program  office  whether  it  was  feasible  to  use 
a  credit  card  program  or  a  combination  of  printed  coupons  and  credit 
cards  to  meet  the  need,  as  agency  officials  stated  that  DLA  has  done  in 
other  countries,  but  the  requirement  was  left  unchanged  and  the  contract 
was  awarded  to  the  single  vendor  that  submitted  an  offer.  Contracting 
officials  told  us  that  as  of  May  2014,  they  were  considering  developing  a 
business  case  analysis  to  assess  the  costs  and  benefits  of  switching  to  a 
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credit  card  program.  DLA  officials  told  us  that  formally  analyzing 
alternative  approaches  and  changing  from  a  coupon  to  a  credit  card 
program  would  need  to  be  coordinated  through  the  Italian  government 
since  tax-free  privileges  are  a  benefit  provided  by  the  Italian  government. 


Market  Research  on 
Lower  Dollar  Value 
Contracts  Was  Often 
Limited  and  May  Have 
Missed  Opportunities  to 
Promote  Competition 


Lower  dollar  value  contracts  we  selected  were  less  likely  to  be  competed 
than  higher  dollar  value  contracts — while  6  of  the  1 6  lower  dollar  value 
contracts  were  competed,  3  of  those  received  only  one  offer,  and  the 
other  10  contracts  were  sole-sourced.  As  noted  previously,  market 
research  is  to  be  used  to  promote  and  provide  for  full  and  open 
competition  or,  when  not  required,  to  obtain  competition  to  the  maximum 
extent  practicable.  Even  in  situations  when  agency  officials  anticipate 
awarding  a  sole-source  contract,  the  contracting  officer  is  generally 
required  to  identify  actions,  if  any,  the  agency  may  take  to  remove  or 
overcome  any  barriers  to  competition  for  future  acquisitions  of  the 
supplies  or  services  required.  Overall,  we  identified  limitations  in  the 
market  research  supporting  seven  lower  dollar  value  contracts  awarded 
by  DOD  and  DHS,  all  of  which  were  sole-sourced  or  received  only  one 
offer.  As  a  result,  DOD  and  DHS  officials  may  have  missed  opportunities 
to  promote  competition.  In  several  cases,  agency  officials  relied  on 
market  research  that  appeared  to  be  incomplete  or  outdated.  Examples  of 
these  cases  are  highlighted  below: 

•  Market  research  on  a  $1.6  million  DHS  contract  to  rent  parking 
spaces  near  a  DHS  building  located  in  Arlington,  Virginia,  cited 
multiple  potential  vendors  through  a  review  of  prior  acquisition  history, 
a  government  database,  and  a  DHS  source  list,  but  the  market 
research  did  not  include  any  formal  outreach  to  industry.  The  market 
research,  conducted  by  a  program  official,  identified  three  potential 
parking  facilities  within  one  block  of  the  DHS  building  but  did  not 
assess  whether  potential  vendors  could  provide  direct  elevator  access 
to  the  building,  a  key  requirement  specifically  included  in  the 
solicitation.  The  agency  received  only  one  offer  from  a  parking  facility 
adjacent  to  the  building.  Subsequently,  an  agency  attorney  who 
reviewed  the  scope  of  work  wrote  that  the  requirements  seemed 
unusually  narrow  and  that  only  one  vendor  would  likely  be  able  to 
provide  a  parking  garage  with  an  elevator  that  connected  to  the  office 
building.  We  found  no  evidence  that  DHS  used  market  research  to 
reassess  whether  DHS’s  requirement  could  be  modified,  and  as  a 
result,  the  agency  may  have  missed  opportunities  to  receive  more 
than  one  offer. 
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•  In  a  $2  million  Army  contract  for  flight  simulator  training  services 
receiving  only  one  offer,  the  primary  market  research  was  a  sources 
sought  notice  posted  for  10  days  on  FedBizOpps  for  which  a 
response  was  only  received  from  the  incumbent  vendor.  Further,  over 
a  10-year  procurement  history  for  these  services,  market  research 
consisted  of  two  sources  sought  notices — the  one  already  discussed 
and  one  issued  in  2006.  While  the  contracting  officer  doubted  whether 
any  other  vendors  had  the  capability  to  fulfill  this  particular 
requirement,  the  contracting  officer  acknowledged  more  market 
research  could  have  been  done  to  ensure  this  conclusion  was 
accurate,  such  as  outreach  to  other  potential  vendors  who  provide 
flight  simulator  services. 

•  Despite  predating  the  formation  of  DHS  and  agency  officials’ 
acknowledgement  of  technological  advances,  the  market  research  for 
a  $627,000  DHS  contract  for  satellite  airtime  needed  to  receive  global 
positioning  data  from  tracking  devices  included  in  CBP  aircraft  had  not 
been  updated  since  1999.  According  to  agency  officials,  the  satellite 
airtime  was  tied  to  the  use  of  those  particular  tracking  devices. 

Agency  officials  indicated  they  reviewed  the  procurement’s  acquisition 
history  and  issued  a  notice  of  intent  to  sole-source,  though  the  notice 
was  issued  after  the  agency  had  already  awarded  the  contract.  Since 
the  government  was  required  to  use  the  same  vendor  for  both  the 
airtime  and  the  tracking  devices,  contracting  officials  deemed  a 
vendor  who  responded  to  the  notice  ineligible.  Contracting  officials  we 
spoke  with  acknowledged  that  the  market  research  was  limited  and 
more  needed  to  be  done  to  assess  whether  there  were  viable 
alternatives  to  the  tracking  devices  and  its  proprietary  satellite  airtime. 

In  two  cases,  while  a  lack  of  technical  data  rights  may  have  contributed, 
in  part,  to  challenges  competing  a  contract,  we  nevertheless  found 
limitations  in  the  market  research  conducted  to  inform  the  acquisition 
approach.^''  For  example: 


^^With  regards  to  the  issue  of  a  lack  of  technical  data  rights,  DLA  officials  noted  that  DLA 
is  responsible  for  sustainment  of  weapons  systems  and  does  not  have  data  rights  for 
many  items  they  procure.  DLA  officials  stated  that  if  potential  alternate  sources  are  not 
found  during  market  research  prior  to  issuing  a  solicitation,  then  DLA  Directive  clause 
52.217-9002  “Conditions  for  Evaluation  of  Offers  for  Part  Numbered  Items”  may  be 
included  in  the  solicitation.  This  clause  may  be  used  in  negotiated  acquisitions  of 
replacement  parts,  components,  and  assemblies  which  are  identified  only  by  the  name  of 
the  approved  source,  a  part  number,  and  a  brief  description,  and  the  clause  provides, 
among  other  things,  that  vendors  can  offer  an  alternate  product,  which  could  increase 
competition.  DLA  Directive  §  §1 7.7501  (b)(3)(i);  52.217-9002. 
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•  In  a  $589,000,  sole-source  DLA  contract  for  actuators,  the  market 
research  consisted  of  a  presolicitation  notice  that  was  issued  2 
months  before  the  contract  was  signed,  an  examination  of  prior 
acquisition  history  with  the  incumbent,  and  a  market  survey,  which 
was  sent  only  to  the  incumbent,  who  did  not  fill  it  out.  Although 
contracting  officials  told  us  that  the  solicitation  was  published  on 
DLA’s  site  for  contracting  opportunities,  they  also  stated  that  there 
were  few  options  for  conducting  market  research  because  the  product 
was  designed  to  adhere  to  specific  government  requirements.  In 
addition,  officials  stated  that  they  typically  rely  on  a  program  office  to 
provide  information  about  whether  a  vendor  is  an  approved  source, 
especially  in  cases  where  the  government  does  not  own  the  data 
rights.  Contracting  officials  also  told  us  that  this  vendor  had  a  history 
of  non-responsiveness  and  they  had  not  expected  to  gather  any 
information  from  the  market  survey. 

•  In  a  $694,000  DLA  contract  for  controller  monitors,  which  are  one  part 
in  a  piece  of  equipment  used  to  provide  oxygen  to  crew  members  in 
an  aircraft,  DLA  faced  a  similar  challenge  because  it  did  not  own  the 
data  rights.  Our  contract  review  revealed  that  contracting  officials  had 
identified  two  sources  but  determined  that  one  of  the  two  sources  was 
not  viable  because  it  was  a  reseller  that  would  obtain  the  item  from 
the  manufacturer;  contracting  officials  told  us  that  they  did  not  pursue 
this  source  because  they  thought  that  the  vendor  would  likely  charge 
a  higher  price  after  obtaining  it  from  the  manufacturer.  The  other  main 
source  of  market  research  cited  in  the  contract’s  justification  to  sole- 
source  was  issuance  of  a  notice  of  intent  to  sole-source,  where  no 
other  vendors  responded. 

While  market  research  on  the  lower  dollar  value  contracts  we  reviewed 
was  often  limited,  this  was  not  always  the  case,  even  in  situations  where 
the  contract  was  not  competed.  For  example,  our  work  found  that  the 
market  research  on  two  lower  dollar  value,  sole-source  contracts  included 
in-depth  analysis  of  potential  sources  found  in  the  market,  identification  of 
reasons  why  the  vendor  was  uniquely  qualified  to  meet  the  need,  and  the 
tradeoffs  that  would  need  to  be  considered  to  compete  the  requirement. 

In  one  procurement  involving  a  $1 .1  million  Army  contract  for  armored 
windows  used  on  mine  clearance  vehicles,  the  market  research  assessed 
three  vendors  and  analyzed  tradeoffs  and  costs  given  that  the 
government  did  not  own  the  data  rights  for  the  windows  currently  used. 
Specifically,  the  market  research  explained  how  the  contracting  officer 
used  Internet  searches  to  identify  specific  vendors,  listed  the  vendors  and 
websites  accessed,  and  used  this  information  to  analyze  tradeoffs 
associated  with  these  potential  sources.  The  market  research  also 
referenced  attempts  to  gauge  industry  interest  by  issuing  a  sources 
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sought  notice  over  6  months  prior  to  issuing  an  intent  to  sole-source 
notice.  The  analysis  quantified  the  time  and  cost  associated  with  reverse 
engineering  the  product  to  produce  a  set  of  government-owned  data 
rights,  which  could  be  used  to  qualify  new  potential  sources.  The 
justification  and  approval  also  noted  that  an  additional  vendor  had 
expressed  interest  in  becoming  a  qualified  source  but  the  agency 
concluded  that  the  vendor  was  unsuitable  for  the  current  procurement 
due  to  the  significant  amount  of  time  needed  to  develop  a  data  package. 


Basic  Market  Research 
Elements  Were  Not 
Consistently  Documented, 
Limiting  Use  for  Future 
Procurements 


Our  review  found  that  all  28  contracts  included  some  evidence  of  market 
research,  but  acquisition  staff  did  not  consistently  document  four  basic 
elements  in  the  contract  file — the  methods  used,  including  the  timeframes 
when  staff  used  them,  an  analysis  of  the  capabilities  of  potential  sources, 
and  a  conclusion  based  on  this  analysis — which  GAO  determined  would 
allow  those  not  connected  with  the  research  to  understand  the  market 
research  conducted. Overall,  14  of  the  28  contracts  we  reviewed 
included  all  of  these  elements  in  the  market  research  documentation  in 
the  contract  files.  However,  while  agency  guidance  describes  market 
research  documentation  as  useful  for  current  and  future  acquisition 
decision  making,  we  could  not  find  one  or  more  of  the  basic  elements  for 
14  of  the  28  contract  files  we  reviewed,  with  10  of  the  14  contracts  being 
ones  we  categorized  as  lower  value.  Nine  of  these  10  contracts  were 
awarded  by  DOD  and  DHS  (see  figure  4). 


op 

In  the  absence  of  guidance  from  DOT,  we  compared  the  contents  of  DOD,  DHS,  and 
FAA  guidance  to  identify  the  minimum  elements  which,  if  recorded  in  the  contract  file, 
would  allow  others,  such  as  reviewers  or  those  procuring  similar  items  in  the  future,  to 
understand  the  nature  of  the  market  research  conducted,  potential  sources,  and  how  it 
was  used. 
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Figure  4:  Number  of  Contracts  Whose  Documentation  inciudes  or  Exciudes  Basic 
Eiements 


Agency 

Documentation  included 
all  elements 

Documentation  excluded 
at  least  one  element 

Department  of  Defense 

r 

Department  of  Homeland 
Security 

1 

Department  of  Transportation 
Federal  Aviation  Administration 

Department  of  Transportation 
Federal  Highway  Administration 

n 

Total:  14  contracts 

Total:  14  contracts 

□ 

□ 


Higher  value  contract  (above  $10  million) 

Lower  value  contract  (between  $500,000  and  $10  million) 


Source:  GAO  analysis  of  DOD,  DHS,  DOT,  and  FAA  contract  documentation.  |  GAO-15-8 


Overall,  8  of  the  12  higher  dollar  contracts  we  reviewed  contained  each  of 
the  basic  elements,  including  information  on  potential  vendors  and 
analyses  of  their  capabilities,  information  more  typically  missing  from  the 
lower  dollar  value  contracts. This  was  true  whether  the  information  was 
contained  in  an  acquisition  plan,  the  justification  for  other  than  full  and 
open  competition,  or  the  market  research  report.  For  example,  for  the 
Army’s  $28.5  million  contract  for  the  procurement  of  advisory  and 
assistance  services,  the  written  acquisition  plan  included  the  names  of 
potential  small  business  vendors  who  responded  to  the  survey  and  an 
assessment  of  their  capabilities.  In  another  example,  for  an  FAA 
procurement  of  a  $126.6  million  engine  generator  set  program,  the 
market  research  memo  recorded  the  names  of  each  vendor  who 
responded  to  a  market  survey  and  an  evaluation  of  their  capabilities, 
including  whether  the  vendor  had  the  capacity  to  meet  the  requirement 
with  in-house  resources. 


oo 

In  response  to  GAO’s  finding  that  one  of  FHWA’s  higher  dollar  contracts  did  not  include 
documentation  of  all  four  elements,  FHWA  officials  told  us  that  they  considered  the  market 
research  conducted  on  the  $20  million  contract  for  legal  advisory  services  to  be  extensive, 
noting  that  as  a  part  of  market  research  the  agency  reached  out  to  dozens  of  potential 
sources.  The  officials  acknowledged,  however,  that  better  documentation  of  the  analysis 
of  the  capabilities  of  these  sources  would  have  strengthened  the  overall  market  research. 
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The  market  research  documentation  on  the  lower  dollar  contracts  was 
more  likely  to  be  missing  at  least  one  element,  particularly  at  DOD  and 
DHS,  where  only  3  of  the  12  lower  dollar  contracts  we  reviewed  included 
documentation  of  all  four  elements.  In  particular,  lower  dollar  value 
contracts  were  less  likely  to  include  information  on  potential  vendors  and 
an  assessment  of  their  capabilities.  For  example,  the  justification  for  other 
than  full  and  open  competition  on  CBP’s  $609,000  sole-source 
procurement  for  portable  receiver  devices  awarded  in  September  2012 
lacked  information  describing  the  capabilities  of  other  possible  vendors 
and  the  dates  on  which  any  market  research  was  conducted.  The 
contract  specialist  explained  that  the  release  of  funding  late  in  the  fiscal 
year  sometimes  prevents  staff  from  drawing  up  more  complete  market 
research  documentation.  In  another  example,  the  acquisition  plan  for  a 
$2.3  million  DLA  contract  for  the  lease  of  office  copy  machines  lacked 
information  about  the  capabilities  of  specific  vendors.  While  the 
acquisition  plan  concluded  that  “at  least  five  vendors  could  supply  the 
equipment  and  associated  services,”  it  did  so  without  identifying  the 
market  research  techniques  used  or  the  names  of  the  potential  vendors. 

In  response  to  our  questions,  the  contracting  office  was  aware  of  the 
names  of  the  five  vendors,  explaining  that  prior  information  and  expertise 
were  relied  upon  due  to  the  non-complexity  of  the  requirement. 

Not  fully  documenting  basic  elements  of  market  research  limits  the 
information  and  insights  that  can  be  carried  forward  for  use  with 
subsequent  contracts,  especially  given  the  potential  for  staff  turnover  in 
the  acquisition  workforce  and  the  possibility  that  the  need  for  the 
requirements  will  recur.  For  example,  contracting  and/or  program  staff  for 
5  of  the  20  selected  contracts  we  reviewed  at  DOD  and  DHS  advised  us 
that  the  original  program  or  contracting  staff  were  no  longer  available  to 
respond  to  questions,  and  in  two  of  these  cases  current  staff  could  not 
fully  respond  to  our  questions.  As  we  previously  concluded,  documenting 
information  is  particularly  important  given  the  frequent  turnover  in  the 
acquisition  workforce. Further,  according  to  agency  officials,  over  three- 
quarters  of  the  20  contracts  we  selected  at  DOD  and  DHS  provide  for 
requirements  that  are  likely  to  recur  in  the  future,  and  therefore, 
adequately  documented  market  research  could  prove  useful  in  the  future. 
As  we  noted  earlier,  requiring  that  these  basic  elements  be  documented 
in  contract  files  would  be  consistent  with  internal  control  standards  that 


^■^GAO-1 1-672. 
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transactions  and  other  significant  events  need  to  be  clearly  documented 
and  the  documentation  should  be  readily  available  for  examination  so  as 
to  ensure  that  management  directives  are  carried  out.^® 


Market  Research  Helped 
Inform  Price 
Reasonableness 
Determinations  in  Most  of 
the  Contracts  Reviewed 


The  FAR  identifies  several  techniques  that  can  be  used  to  help  determine 
that  a  price  is  fair  and  reasonable,  including  a  comparison  between 
proposed  prices  received  in  response  to  a  solicitation  or  a  comparison 
between  proposed  prices  and  the  independent  government  cost 
estimate.^®  While  not  required  by  the  FAR  or  agency  guidance,  agencies’ 
guidance  recognizes  the  usefulness  of  market  research  in  assessing  what 
price  would  be  reasonable  for  the  government  to  pay  for  a  procurement. 

In  addition,  0MB  has  stated  that  outreach  to  vendors  is  important  so  that 
the  government  has  a  clear  understanding  of  the  market  place  and  can 
award  a  contract  for  an  effective  solution  at  a  reasonable  price. 
Contracting  officers  cited  one  or  more  market  research  techniques  as 
contributing  to  their  determination  of  a  fair  and  reasonable  price  in  21  of 
the  28  contracts  we  reviewed.  The  market  research  for  5  of  these 
contracts  involved  communication  with  industry. 


We  identified  the  use  of  a  variety  of  market  research  techniques  in 
determining  price  reasonableness  such  as  a  review  of  published  price 
lists,  reviewing  historical  prices,  and  techniques  involving  communication 
with  industry.  While  using  market  research  to  help  inform  the  price 
reasonableness  determination  tended  not  to  involve  communication  with 
industry,  5  contracts  we  reviewed  did  cite  market  research  techniques 
that  involved  communication  with  industry.  The  techniques  involving 
communication  with  industry  included  obtaining  prices  through  requests 
for  information  posted  on  FedBizOpps,  vendor  quotes,  and  government 
requests.  Two  of  the  five  were  higher  dollar  contracts — one  was 
competed  and  received  multiple  offers  and  the  other  was  sole-sourced — 
where  the  contracting  officer  used  pricing  information  received  from 
industry  in  response  to  government  requests  for  information  to  determine 
price  reasonableness.  For  two  of  the  three  lower  dollar  contracts,  the 
market  research  technique  used  was  to  obtain  quotes  from  other  vendors 


®®GAO/AIMD-00-21.3.1. 

®®The  contracting  officer  may  use  an  independent  government  cost  estimate  to  help 
establish  the  reasonableness  of  contractors’  costs  or  prices. 

®^See  0MB,  “Myth-Busting”  Memorandum  (Washington,  D.C.:  Feb.  2,  2011). 
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to  compare  to  the  proposal  received — both  of  these  were  competed 
contracts  where  only  one  offer  was  received. 

In  5  of  the  21  cases  where  contracting  officers  cited  ways  that  market 
research  helped  in  the  determination  of  fair  and  reasonable  prices, 
contracting  officers  also  cited  how  this  pricing  information  helped  the 
government  to  negotiate  a  price  lower  than  what  was  originally  proposed 
by  the  vendor.  For  example,  in  the  $210  million  DLA  contract  to  provide 
fuel  in  Italy,  based  on  our  review  of  contract  documentation,  the 
contracting  officer  reviewed  pricing  information  from  the  country’s 
marketplace  through  published  pricelists  and  prior  contract  prices,  and 
used  this  information  to  set  pricing  goals  for  negotiations  with  the  vendor. 
According  to  the  contract  file  documentation,  the  government  was  able  to 
negotiate  a  lower  price  for  certain  types  of  fuel  than  what  the  contractor 
had  originally  proposed  even  though  only  one  offer  was  received.  As  we 
noted  previously,  however,  DLA  did  not  assess  whether  its  requirement  to 
use  a  printed  coupon  system  could  be  modified  to  consider  the  use  of  a 
credit  card  system,  which  may  have  enabled  DLA  to  obtain  the  benefits  of 
market  forces  to  help  negotiate  even  lower  prices. 

In  contrast,  in  one  case  in  which  the  contracting  officer  cited  market 
research  as  supporting  a  fair  and  reasonable  price  determination,  we 
found  that  opportunities  to  use  additional  market  research  or  other  price 
reasonableness  techniques  may  have  been  missed.  For  a  3-year,  $1 .5 
million  sole-source  contract  awarded  by  DHS  for  membership  services  to 
an  organization  that  oversees  the  creation  and  use  of  business 
guidelines,  agency  officials  were  unable  to  provide  documentation  based 
on  market  research  or  other  techniques  to  fully  support  their  price 
reasonableness  decision.  The  justification  and  approval  stated  that  the 
contracting  officer  had  determined  the  anticipated  prices  to  be  fair  and 
reasonable  based  on  price  comparisons  to  the  prior  contract,  and  by 
comparing  a  published  price  list  against  the  government’s  independent 
government  cost  estimate.  However,  DHS  officials  admitted  that  the 
scope  of  work  for  the  previous  contract  was  different  from  the  scope  of 
work  for  the  current  contract  and  that  the  contracts  could  not  be  fully 
compared.  In  addition,  we  found  that  the  price  of  annual  membership 
services  under  the  current  contract  and  as  cited  in  its  independent 
government  cost  estimate  was  more  than  1 0  times  higher  than  the  price 
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for  full  membership  services  as  listed  on  the  published  price  list.^® 
Subsequently,  a  program  official  provided  us  with  a  high  level  breakdown 
of  the  costs  that  had  been  provided  to  the  program  office  by  the  vendor 
prior  to  contract  award  but  advised  us  that  they  had  no  additional 
documentation  to  support  the  price  reasonableness  determination  within 
the  program  office. 

For  the  7  contracts  that  did  not  use  market  research  techniques  to 
determine  price  reasonableness,  the  contracting  officers  indicated  that 
they  used  techniques  such  as  comparing  proposed  prices  to  the 
independent  government  cost  estimate  and  comparing  prices  among 
proposals  received.  For  example,  most  of  the  competitive  contracts  we 
reviewed  that  received  multiple  offers  cited  the  comparison  of  offers 
received  as  contributing  to  their  determination  of  a  fair  and  reasonable 
price.  In  addition,  a  contracting  officer  is  required  to  obtain  certified  cost 
or  pricing  data  from  the  contractor  in  certain  instances,  including  for 
negotiated  contracts  above  $700,000,  unless  an  exception  applies,  such 
as  obtaining  adequate  price  competition  or  acquisitions  of  a  commercial 
item.^®  Nearly  all  of  the  noncompeted  contracts  we  reviewed  were  either 
below  the  $700,000  threshold  for  which  certified  cost  or  pricing  data  may 
be  required,  or  were  for  commercial  items,  and  therefore  did  not  obtain 
certified  cost  and  pricing  data.  In  the  two  cases  in  which  contracting 
officers  were  required  to  obtain  certified  cost  and  pricing  data,  we  found 
acquisition  officials  did  so. 


Conclusions 


Federal  regulations  and  the  agency  guidance  we  reviewed  highlighted  the 
need  to  conduct  and  document  market  research  commensurate  with  the 
proposed  acquisitions’  dollar  value  and  complexity.  We  found  evidence  of 
market  research  in  all  28  contracts  we  reviewed.  Similarly,  we  often  found 
that  acquisition  officials  conducted  more  robust  market  research  on 
higher  dollar  value  contracts  in  particular  by  communicating  with  industry 
about  their  requirements  and  needs,  such  as  through  the  use  of  requests 


oo 

A  program  official  told  us  that  the  published  price  list  reflected  a  basic  membership  price 
and  the  total  contract  price  included  the  price  for  additional  membership  services  outlined 
in  the  statement  of  work,  such  as  access  to  international  standards,  workshops  convened 
by  a  panel  focused  on  the  development  and  adoption  of  standards  critical  to  homeland 
security,  and  a  database.  However,  there  was  no  documentation  to  support  the  price 
reasonableness  of  these  additional  services  included  in  the  contract  file. 

®®FAR  §15.403-4. 
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for  information.  In  turn,  this  market  research  appeared  to  contribute  to 
higher  degrees  of  competition  for  higher  value  contracts.  Our  findings 
also  suggest,  however,  that  agencies  may  be  missing  opportunities  to 
promote  competition  and  inform  future  acquisitions  for  lower  dollar 
acquisitions.  In  this  regard,  we  found  DOD  and  DHS  acquisition  officials 
often  relied  on  internal  sources  of  information  or  simply  notified  potential 
vendors  that  they  intended  to  award  contracts  on  a  sole-source  basis 
rather  than  gauging  whether  the  marketplace  could  offer  an  alternative, 
competitive  solution.  Even  in  situations  where  the  government  anticipates 
a  sole-source  award,  market  research  is  a  tool  that  is  required  to  be  used 
to  promote  competition  to  the  maximum  extent  practicable,  and  agencies 
generally  must  identify  actions,  if  any,  they  may  take  to  remove  or 
overcome  barriers  to  competition  for  future  acquisitions  of  the  supplies  or 
services  required.  We  also  found  instances  in  which  the  market  research 
appeared  to  be  incomplete  or  outdated. 

Further,  we  found  that  acquisition  officials  omitted  one  or  more  of  four 
basic  elements  of  market  research  which  GAO  identified  as  necessary  to 
enable  others  to  understand  the  research — the  methods  used,  the 
timeframes  when  it  was  conducted,  an  analysis  of  vendor  capabilities, 
and  a  conclusion  based  on  this  analysis — in  14  of  the  28  contracts  we 
reviewed.  These  documentation  shortfalls  were  more  often,  though  not 
exclusively,  found  in  lower  value  contracts.  Despite  DOD  and  DHS  having 
guidance  that  either  requires  or  recommends  that  specific  elements  be 
documented,  contracts  from  these  agencies  accounted  for  all  but  one  of 
the  10  lower  value  contracts  missing  one  or  more  of  the  elements.  While 
DOT  documented  these  four  elements  in  most  of  the  contract  files  we 
reviewed,  DOT’S  market  research  guidance  does  not  address  whether 
these  four  basic  elements  should  be  documented.  As  a  result,  DOT  is  at 
increased  risk  that  it  will  not  document  the  basic  information  needed  to 
help  inform  current  and  future  procurements.  Federal  internal  control 
standards  state  that  transactions  and  other  significant  events  need  to  be 
clearly  documented  and  the  documentation  should  be  readily  available  for 
examination,  so  as  to  ensure  management  directives  are  carried  out. 
Moreover,  given  the  recurring  nature  of  the  government’s  requirements 
covered  in  the  contracts  we  reviewed  and  the  staff  turnover  we  observed 
during  the  course  of  our  work,  not  clearly  documenting  such  fundamental 
information  hinders  the  use  of  market  research  to  inform  future 
acquisitions. 
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Rprnmmpnrlptinnci  fnr  ensure  that  key  market  research  information  is  available  for 

cUU  c  Udl  U  b  U  current  and  future  use,  we  recommend  that  the  Secretary  of 

Executive  Action  Transportation  update  its  market  research  guidance  to  specify,  at  a 

minimum,  that  four  basic  elements — the  market  research  methods  used, 
the  timeframes  when  staff  used  them,  an  analysis  of  the  capabilities  of 
potential  sources,  and  a  conclusion  based  on  this  analysis — be 
documented. 

To  help  promote  competition,  inform  future  acquisitions,  and  encourage 
the  use  of  timely  and  complete  market  research,  we  recommend  that  the 
Secretaries  of  Defense  and  Homeland  Security  take  action  to  ensure  their 
acquisition  personnel  clearly  document  the  basic  elements  of  the  market 
research  that  was  conducted.  More  specifically,  documentation  should,  at 
a  minimum,  include  the  market  research  methods  used,  the  timeframes 
when  staff  used  them,  an  analysis  of  the  capabilities  of  potential  sources, 
and  a  conclusion  based  on  this  analysis. 


Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  this  report  to  DOD,  DHS,  and  DOT  for  comment. 

All  three  agencies  concurred  with  our  recommendations.  The  agencies’ 
comments  are  summarized  below.  Written  comments  from  DOD,  DHS, 
and  DOT  are  reproduced  in  appendixes  II,  III,  and  IV  respectively.  DHS 
and  DOT  provided  technical  comments  which  we  incorporated  into  the 
report  as  appropriate. 

DOD  agreed  with  the  recommendation  that  it  take  action  to  ensure  its 
acquisition  personnel  clearly  document  the  basic  elements  of  market 
research  conducted,  but  did  not  identify  specific  actions  it  intends  to  take. 
DOD  noted  that  it  recently  issued  a  memorandum  highlighting  the  need 
for  effective  market  research  to  increase  competition. For  example,  the 
memorandum  notes  that  contracting  officers  will  be  required  to  use 
requests  for  information  or  sources  sought  notices  for  certain 
procurements  where  a  non-competitive  contract  is  contemplated. 

DHS  also  agreed  with  the  recommendation  that  it  take  action  to  ensure  its 
acquisition  personnel  clearly  document  the  basic  elements  of  market 
research  conducted.  DHS  stated  that  its  Office  of  the  Chief  Procurement 


^°DOD,  Office  of  the  Under  Secretary  for  Acquisition,  Technology,  and  Logistics,  Actions 
to  Improve  Department  of  Defense  Competition  (Washington,  D.C.:  Aug.  21, 2014). 
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Officer  will  update  procedures  in  the  Homeland  Security  Acquisition 
Manual  and  its  market  research  guide  to  clarify  that  market  research 
reports  should  include,  at  a  minimum,  the  four  basic  elements.  Given  that 
we  found  that  several  DHS  contract  files  we  reviewed  did  not  include 
documentation  of  elements  already  required  by  DHS,  it  will  be  important 
for  DHS  to  ensure  its  contracting  personnel  follow  its  guidance. 

DOT  agreed  with  our  recommendation  that  it  update  its  market  research 
guidance  to  identify  documentation  of  the  four  basic  elements,  but  did  not 
elaborate  on  specific  actions  it  intends  to  take.  DOT  stated  that  it 
recognizes  the  important  role  market  research  plays  in  developing  sound 
acquisition  strategies,  promoting  competition,  and  informing  price 
reasonableness.  DOT  noted  that  its  market  research  guidance  is 
consistent  with  the  requirements  of  the  FAR,  and  noted  that  the  agency 
continues  to  seek  opportunities  and  incorporate  best  practices  to  further 
strengthen  departmental  guidance.  Given  the  importance  of  written 
guidance  in  helping  to  ensure  that  management  directives  are 
implemented,  we  continue  to  encourage  DOT  to  update  its  guidance  to 
require  documentation  of  the  four  basic  elements  on  a  timely  basis. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Defense,  the 
Secretary  of  Homeland  Security,  the  Secretary  of  Transportation, 
appropriate  congressional  committees,  and  other  interested  parties.  This 
report  will  also  be  available  at  no  charge  on  GAO’s  website  at 
http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841  or  by  e-mail  at  dinapolit@gao.gov.  Contact 
points  for  our  Offices  of  Congressional  Relations  and  Public  Affairs  may 
be  found  on  the  last  page  of  this  report.  Key  contributors  to  this  report  are 
listed  in  appendix  V. 


Director 

Acquisition  and  Sourcing  Management 
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Our  objectives  were  to  assess  (1)  selected  agencies’  guidance  and  policy 
on  market  research  and  (2)  how  selected  agencies  conducted  market 
research  on  selected  procurements.  Market  research  serves  several 
purposes,  including  identifying  sources  capable  of  meeting  the  need, 
promoting  competition,  identifying  whether  commercial  items  can  meet 
the  government’s  needs,  and  identifying  whether  small  businesses  are 
capable  of  meeting  the  requirement.  For  the  purposes  of  our  review,  we 
focused  our  work  on  how  market  research  informed  the  competition 
strategy  and  the  contracting  officer’s  analysis  of  price  reasonableness. 
We  did  not  assess  how  market  research  informed  the  small  business 
determination  or  how  it  informed  the  commerciality  decision  given  GAO 
has  a  prior  body  of  work  that  focuses  on  these  issues.”' 

To  assess  agency  guidance  and  policy  on  market  research,  we  reviewed 
government-wide  regulations  and  policy,  such  as  the  Federal  Acquisition 
Regulation  (FAR),  including  FAR  Subpart  7.1 — Acquisition  Plans  and 
FAR  Part  10 — Market  Research,  and  Office  of  Management  and  Budget 
(0MB)  memorandums  related  to  market  research,  such  as  OMB’s  Office 
of  Federal  Procurement  Policy’s  memorandum  entitled,  “Myth-Busting”: 
Addressing  Misconceptions  to  Improve  Communication  with  Industry 
During  the  Acquisition  Process.  To  assess  specific  agency  guidance  and 
policy  and  to  illustrate  how  market  research  is  performed  at  agencies  of 
various  sizes  and  that  procure  a  range  of  products  and  services,  we 
selected  three  departments  for  our  review — the  Departments  of  Defense 
(DOD),  Homeland  Security  (DHS)  and  Transportation  (DOT).  We 
selected  these  departments  based  on  their  level  of  fiscal  year  2012 
contract  obligations — the  most  recent  fiscal  year  of  information  available 
when  we  started  our  review — and  on  the  mix  of  products  and  services 
procured.  Within  each  department,  we  selected  two  components  for 
further  review — DOD’s  Department  of  the  Army  and  Defense  Logistics 
Agency  (DLA);  DHS’s  Office  of  Procurement  Operations  (OPO)^  and  U.S. 
Customs  and  Border  Protection  (CBP);  and  DOT’S  Federal  Aviation 


Vor  GAO’S  recent  work  on  commercial  item  determinations,  see  GAO,  Federal 
Contracting:  Commercial  Item  Test  Program  Beneficial,  but  Actions  Needed  to  Mitigate 
Potential  Risks,  GAO-14-178  (Washington  D.C.:  Feb.  4,  2014).  For  GAO’s  recent  work  on 
small  business  contracting,  see  GAO,  Small  Business  Contracting:  Updated  Guidance 
and  Reporting  Needed  for  Consolidated  Contracts,  GAO-14-36  (Washington  D.C.:  Nov. 
26,  2013). 

^For  the  purposes  of  this  report,  we  considered  DFIS’s  Office  of  Procurement  Operations 
to  be  a  component. 
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Administration  (FAA)  and  Federal  Highway  Administration  (FHWA).  In 
general,  we  selected  the  components  with  the  largest  fiscal  year  2012 
obligations  based  on  the  Federal  Procurement  Data  System — Next 
Generation  (FPDS-NG).  For  each  department  and  component,  as 
applicable,  we  reviewed  guidance  and  policy  related  to  market  research, 
including  agency  supplements  to  the  FAR,  market  research  guides  and 
templates,  acquisition  handbooks,  and  policy  memorandums  that 
reference  market  research,  such  as  DOD’s  Better  Buying  Power  policy 
memorandums.  Key  agency  guidance  reviewed  included  DOD’s  Defense 
Standardization  Program,  Market  Research:  Gathering  Information  about 
Commercial  Products  and  Services,  SD-5;  DHS’s  Market  Research 
Guide;  DOT’S  Market  Research  Guide,  Acquisition  Policy  DOT  Dash 
2010-11;  and  FAA’s  Procurement  Guidance  T3.2.1.2  -  Market  Analysis.^ 
The  FAR  and  DOT  acquisition  regulation  do  not  apply  to  FAA  acquisitions 
so  we  analyzed  FAA’s  policy  and  guidance  separately  from  DOT.'*  We  did 
not  include  agency  training  materials  as  guidance  for  the  purposes  of  our 
analyses.  Further,  we  interviewed  senior  acquisition  officials  at  each 
department  to  develop  a  better  understanding  of  agency  policies, 
guidance,  and  initiatives  related  to  market  research. 

To  assess  how  agencies  conducted  market  research  on  selected 
procurements,  we  selected  and  reviewed  a  nongeneralizable  sample  of 
28  contracts.  For  each  contract,  we  reviewed  contract  documents  related 
to  market  research,  including  market  research  reports,  acquisition  plans, 
justifications  and  approvals  for  sole-source  awards,  price  negotiation 
memorandums  and  independent  government  cost  estimates.  We 
analyzed  these  documents  to  identify  market  research  methods,  analysis, 
and  results  used  to  inform  the  competition  strategy  and  to  inform  the  price 
reasonableness  determination.  To  identify  information  that  may  not  have 
been  included  or  been  self-explanatory  based  on  our  review  of  the 
contract  files,  we  obtained  additional  information  from  agency  officials. 

Our  inquiries  largely  focused  on  identifying  market  research  techniques 
that  agency  officials  may  have  used  but  were  not  reflected  in  the  contract 


^While  this  review  focused  on  guidance  applicable  across  each  department  and 
separately  for  FAA,  FHWA  noted  its  Acquisition  Customer  Service  Guide  contains 
additional  information  on  market  research  techniques. 

"^For  the  purposes  of  this  report,  we  use  the  term  agency  policy  and  guidance  to  refer  to 
that  issued  by  DOD,  DHS,  DOT,  as  well  as  FAA.  The  FAR  does  not  apply  to  FAA 
acquisitions  pursuant  to  the  Department  of  Transportation  and  Related  Agencies 
Appropriations  Act,  1996.  Pub.  L.  No.  104-50,  §  348  (1995). 


Page  34 


GAO-15-8  Market  Research 


Appendix  I:  Objectives,  Scope,  and 
Methodology 


file,  as  well  as  obtaining  further  explanations  of  any  analyses  they  may 
have  performed.  We  also  analyzed  contract  documentation  to  determine 
whether  these  documents  included  four  basic  elements  we  identified 
based  on  our  review  of  agency  policy  and  guidance,  which,  if  recorded, 
would  allow  those  not  connected  with  the  market  research  to  obtain  at 
least  a  basic  understanding  of  how  the  acquisition  team  collected  and 
analyzed  information  about  market  capabilities.  We  also  identified  these 
elements  because  they  focused  on  collection  and  analysis  of  information, 
consistent  with  the  FAR’s  definition  of  market  research.^  The  four 
elements  included  the  market  research  methods  used,  including  the 
timeframes  when  acquisition  staff  used  them,  an  analysis  of  the 
capabilities  of  potential  sources,  and  a  conclusion  based  on  this  analysis. 

To  identify  contracts  for  our  review,  we  used  FPDS-NG  to  select  a 
nongeneralizable  sample  of  30  contracts  awarded  in  fiscal  year  2012  from 
the  six  components  within  DOD,  DHS,  and  DOT.  As  our  review  did  not 
focus  on  how  market  research  informed  the  small  business  set-aside 
determination  or  the  commercial  item  determination,  we  excluded 
contracts  set  aside  for  small  businesses  and  contracts  for  research, 
development,  test,  and  evaluation.  To  increase  the  comparability  of 
contracts  across  agencies,  we  excluded  weapons  systems  contracts,  as 
they  are  unique  to  DOD.  We  also  excluded  contracts  with  no  fiscal  year 
2012  obligations  and  contracts  valued  at  less  than  $500,000,  selecting 
this  threshold  as  the  contracts  above  this  threshold  represented  more 
than  92  percent  of  contract  dollars  awarded  in  fiscal  year  2012  at  each 
component  reviewed. 

Within  each  selected  agency  component,  we  created  randomized  lists  of 
the  remaining  fiscal  year  2012  contracts  within  two  strata — (1)  contracts 
valued  at  more  than  $10  million  and  (2)  contracts  valued  between 
$500,000  and  $10  million.  We  selected  two  contracts  from  the  first 
stratum  and  three  from  the  second  stratum  (five  per  component)  from  the 
randomized  lists  for  each  component.  Whereas  the  higher  dollar  stratum 
represented  the  greatest  portion  of  contract  dollars  awarded  in  fiscal  year 
2012  within  each  component — ranging  between  79  percent  and  98 
percent  of  dollars — the  lower  dollar  stratum  represented  a  larger  portion 
of  contract  actions  awarded  that  year.  Within  each  stratum,  we  selected 


^FAR  §  2.101  defines  market  research  as  the  process  used  to  collect  and  analyze  data 
about  capabilities  in  the  market  that  could  satisfy  an  agency’s  procurement  needs. 
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the  contracts  starting  from  the  top  of  the  randomized  lists  but  applying 
criteria  to  ensure,  to  the  extent  possible,  that  our  selected  contracts  within 
each  component  included  at  least  one  contract  for  products  and  one  for 
services,  at  least  one  competed  and  one  noncompeted  contract,  at  least 
one  competed  contract  receiving  one  offer  and  one  competed  contract 
receiving  multiple  offers,  among  other  factors. 

To  assess  the  reliability  of  the  FPDS-NG  data  used  to  select  the  contracts 
for  review,  we  compared  key  data  elements  from  FPDS-NG  to  the 
contract  documentation,  including  contract  number,  agency  name,  and 
whether  or  not  the  contract  had  been  competed  using  full  and  open 
competition.  Where  we  found  discrepancies  relevant  to  our  methodology, 
we  made  adjustments  to  maintain  the  integrity  of  our  selection  criteria. 

For  example,  we  excluded  7  contracts  because  the  contracts  were 
miscoded  in  FPDS-NG  and  therefore  should  have  been  excluded  from 
our  sample.  Specifically,  we  excluded  5  contracts  because  they  were 
small  business  set  asides,  one  contract  because  the  total  dollar  value  fell 
below  the  $500,000  threshold  for  inclusion  in  the  review,  and  one  contract 
because  it  was  a  research  and  development  contract.  Because  we  had 
randomized  lists  of  contracts  for  each  of  the  components,  we  replaced  the 
miscoded  contracts  with  another  contract.  Once  we  corrected  for  these 
errors,  we  concluded  the  data  obtained  from  the  FPDS-NG  was 
sufficiently  reliable  for  the  purposes  of  this  review. 

We  excluded  2  of  the  30  selected  contracts  from  our  final  analysis — one 
from  FHWA  and  one  from  FAA — because,  as  of  July  31, 2014,  we  were 
unable  to  assess  the  market  research  for  those  contracts  due  to  missing 
market  research  documentation  and  staff  changes  since  the  award  of  the 
contracts.®  As  a  result,  our  analysis  in  this  report  is  based  on  28  selected 
contracts  (see  table  2).  The  findings  from  our  selected  contracts  are  not 
generalizable  to  the  population  of  all  contracts  within  the  selected 
agencies  or  components. 


®At  the  conclusion  of  our  review  in  August  2014,  FHWA  officials  reported  they  had  located 
the  preaward  documents  for  the  contract.  For  the  FAA  contract,  contracting  staff  were 
unable  to  locate  the  market  research  for  the  contract.  In  both  instances  the  staff 
responsible  for  conducting,  documenting,  or  accepting  the  market  research  were  no 
longer  with  the  organization. 
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Table  2:  Description  of  28  Selected  Contracts  Reviewed  by  GAO 


Contract  description 

Agency/Component 

Competition 

Contract  dollar 
value 

Higher  Dollar  Contracts  (above  $10  million) 

Data  communication  integrated  services 

DOT/FAA 

Competed,  multiple  offers 
received 

$2,208,000,000 

Tax-free  fuel 

DOD/DLA 

Competed,  one  offer 
received 

$210,384,000 

Jet  fuel  with  additives 

DOD/DLA 

Competed,  multiple  offers 
received 

$157,272,000 

Engine  generator  set  program 

DOT/FAA 

Competed,  one  offer 
received 

$126,587,000 

Protective  service  officer  services 

DHS/OPO 

Competed,  multiple  offers 
received 

$86,993,000 

Protective  service  officer  services 

DHS/OPO 

Competed,  multiple  offers 
received 

$76,975,000 

Equipment  maintenance  program  support  and  training 
support  (bridge  contract) 

DHS/CBP 

Noncompeted 

$72,419,000 

Coastal  interceptor  vessel 

DHS/CBP 

Competed,  multiple  offers 
received 

$45,719,000 

Advisory  and  assistance  services 

DOD/Army 

Competed,  multiple  offers 
received 

$28,500,000 

Construction  services 

DOT/FHWA 

Competed,  multiple  offers 
received 

$23,360,000 

Legal  advisory  services 

DOT/FHWA 

Competed,  multiple  offers 
received 

$20,234,000 

Construction  services  for  foreign  aid  projects 

DOD/Army 

Competed,  multiple  offers 
received 

$15,000,000 

Lower  Dollar  Contracts  (between  $500,000  and  $10  million) 

Technical  and  program  support 

DOT/FHWA 

Competed,  multiple  offers 
received 

$9,642,000 

Transportation  standards  development  support 

DOT/FHWA 

Noncompeted 

$7,479,000 

Construction  of  utilities  infrastructure 

DOD/Army 

Noncompeted 

$4,047,000 

Lease  of  copiers,  multifunction  devices 

DOD/DLA 

Competed,  multiple  offers 
received 

$2,259,000 

Flight  simulator  training 

DOD/Army 

Competed,  one  offer 
received 

$2,067,000 

Relocation  of  Doppler  navigation  system 

DOT/FAA 

Competed,  multiple  offers 
received 

$1,631,000 

Parking 

DHS/OPO 

Competed,  one  offer 
received 

$1,585,000 

Standards  organization  membership  services 

DHS/OPO 

Noncompeted 

$1,500,000 

Armor,  transparent,  vehicular  window 

DOD/Army 

Noncompeted 

$1,132,000 
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Contract  description 

Agency/Component 

Competition 

Contract  doiiar 
vaiue 

Recycle  and  store  tubes  of  helium 

DHS/OPO 

Noncompeted 

$983,000 

Jail  space  services 

DHS/CBP 

Competed,  one  offer 
received 

$846,000 

Controller  monitors 

DOD/DLA 

Noncompeted 

$694,000 

Satellite  airtime 

DHS/CBP 

Noncompeted 

$627,000 

Portable  receiver  systems 

DHS/CBP 

Noncompeted 

$609,000 

Actuators 

DOD/DLA 

Noncompeted 

$589,000 

Furniture,  workstations 

DOT/FAA 

Noncompeted 

$538,000 

Source:  GAO  analysis  of  agency  documentation.  |  GAO-15-8. 


We  conducted  this  performance  audit  from  August  201 3  to  October  2014 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
3000  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3000 


ACQUISITION. 
TECHNOLOGY 
AND  LOGISTICS 


Mr.  Timothy  J.  DiNapoli 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington.  DC  20548 


SEP  3  G  im 


Dear  Mr.  DiNapoli: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  Draft  Report,  GAO-15-8, 
“MARKET  RESEARCH:  Better  Documentation  Needed  to  Inform  Future  Procurements  at 


Selected  Agencies,”  dated  September  5,  2014  (GAO  Code  121 159).  The  detailed  comments  on 
the  report  recommendation  are  enclosed. 


Richard  Ginman 
Director,  Defence  Procurement 
and  Acquisition/Policy 


Enclosure: 
As  stated 
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Appendix  II:  Comments  from  the  Department 
of  Defense 


GAO  Draft  Report  Dated  September  5,  2014 
GAO-15-8  (GAO  CODE  121139) 

“MARKET  RESEARCH:  BETTER  DOCUMENTATION  NEEDED  TO  INFORM 
FUTURE  PROCUREMENTS  AT  SEUECTED  AGENCIES” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RECOMMENDATION  I :  The  GAO  recommends  that  the  Secretary  of  Defense  take  action  to 
ensure  their  acquisition  personnel  clearly  document  the  basic  elements  of  the  market  research 
that  was  conducted.  More  specifically,  documentation  should,  at  a  minimum,  include  the  market 
research  methods  used,  the  timeframes  when  staff  used  them,  an  analysis  of  the  capabilities  of 
potential  sources,  and  a  conelusion  based  on  this  analysis. 

DoD  RESPONSE:  Concur.  The  Department  of  Defense  (DoD)  will  continue  to  emphasize  the 
importance  of  documenting  market  research  efforts  to  include  these  four  elements.  The  DoD 
recently  highlighted  the  need  for  effective  market  research  in  an  August  21. 2014.  policy 
memorandum  signed  by  the  Under  Secretary  for  Acquisition.  Technology,  and  Logistics  entitled. 
"Actions  to  Improve  Department  of  Defense  Competition." 
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Appendix  III:  Comments  from  the 
Department  of  Homeland  Security 


U.S.  Department  of  Homeland  Security 
Washington,  DC  20528 


Homeland 

Security 


September  24,  2014 


Timothy  J.  DiNapoli 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Re:  Draft  Report  GAO-1 5-8,  “MARKET  RESEARCH:  Better  Documentation  Needed  to 

Inform  Future  Procurements  at  Selected  Agencies” 

Dear  Mr.  DiNapoli: 

Thank  you  for  the  opportunity  to  review  and  comment  on  this  draft  report.  The  U.S.  Department 
of  Homeland  Security  (DHS)  appreciates  the  U.S.  Government  Accountability  Office’s  (GAO) 
work  in  planning  and  conducting  its  review  and  issuing  this  report. 

DHS  is  pleased  to  note  GAO’s  recognition  that  (1)  the  Department  has  market  research  guidance 
in  place  that  is  consistent  with  the  federal  regulation  in  terms  of  market  objectives,  techniques, 
and  documentation,  and  (2)  in  some  instances  DHS  guidance  expands  upon  suggested  federal 
acquisition  regulation  techniques.  DHS  understands  that  market  research  is  a  critical  step  in  the 
acquisition  process,  informing  key  decisions  about  how  best  to  acquire  goods  and  services. 

The  draft  report  contained  one  recommendation  for  DHS,  with  which  the  Department  concurs. 
Specifically,  GAO  recommended  that  the  Secretary  of  Homeland  Security: 

Recommendation:  Take  action  to  ensure  [that]  acquisition  persormel  clearly  document  the 
basic  elements  of  the  market  research  that  was  conducted.  More  specifically,  documentation 
should,  at  a  minimum,  include  the  market  research  methods  used,  the  timeframes  when  staff  used 
them,  an  analysis  of  the  capabilities  of  potential  sources,  and  a  conclusion  based  on  this  analysis. 

Response:  Concur.  While  the  Department’s  Market  Research  guide  cunently  includes  a 
template  that  identifies  the  four  basic  elements  of  market  research,  the  procedures  on 
documenting  market  research  in  the  Homeland  Security  Acquisition  Manual  (HSAM)  and  the 
Market  Research  guide  address  three  of  the  four  elements.  Therefore,  the  DHS  Office  of  the 
Chief  Procurement  Officer  will  update  its  procedures  in  the  HSAM  and  the  Market  Research 
guide  to  clarify  that  market  research  reports  should  include,  at  a  minimum,  the  following  four 
elements:  market  research  methods  used;  the  timeframes  when  staff  used  them;  an  analysis  of 
the  capabilities  of  potential  sources;  and  a  conclusion  based  on  this  analysis.  Estimated 
Completion  Date:  December  31,  2014. 
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Appendix  III:  Comments  from  the  Department 
of  Homeland  Security 


Again,  thank  you  for  the  opportunity  to  review  and  comment  on  this  draft  report.  Technical 
comments  were  previously  provided  under  separate  cover.  We  look  forward  to  working  with 
you  in  the  future. 


Sincerely, 


Jihh  H.  Crumpacker,  CIA,  CFE 
Director 

Departmental  GAO-OIG  Liaison  Office 


2 
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Appendix  IV:  Comments  from  the 
Department  of  Transportation 


U.S.  Department  Assistant  Secretary  1200  New  Jersey  Avenue,  SE 

of  Transportation  for  Administration  Washington,  DC  20590 

Office  of  fhe  Secrefary 
of  Transportation 

OCT  O8  2014 


Timothy  J,  DiNapoli 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street  NW 
Washington,  DC  20548 

The  Department  of  Transporatation  (DOT)  recognizes  the  important  role  market  research  plays 
in  developing  sound  acquisition  strategies,  promoting  competition,  and  informing  price 
reasonableness.  The  Department’s  Market  Research  Guide  is  consistent  with  the  requirements  of 
the  Federal  Acquisition  Regulation  and  we  continue  to  seek  opportunities  and  incorporate  best 
practices  to  further  strenghten  departmental  guidance. 

Upon  preliminary  review,  DOT  concurs  with  GAO’s  recommendation.  The  Department  will 
provide  a  detailed  response  to  the  recommendation  with  in  60  days  of  the  GAO  report  issuance. 

We  appreciate  this  opportunity  to  offer  additional  perspective  on  the  GAO  draft  report.  Please 
contact  Martin  Gertel,  Director  of  Audit  Relations,  at  (202)  366-5145  with  any  questions  or  if 
the  GAO  would  like  to  obtain  additional  details  about  these  comments. 


Assistant  Secretary  for  Administration 
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Appendix  V:  GAO  Contact  and  Staff 
Acknowledgments 


GAO  Contact 

Timothy  J.  DiNapoli,  (202)512-4841  ordinapolit@gao.gov 

Staff 

Acknowledgments 

In  addition  to  the  contact  named  above,  Penny  Berrier,  Assistant  Director; 
Sonja  Bensen,  Jenny  Chanley,  Danielle  Greene,  Kristine  Massinger,  Julia 
Kennon,  Janet  McKelvey,  Guisseli  Reyes-Turnell,  Oziel  Trevino,  and 
Jocelyn  Yin  made  key  contributions  to  this  report. 
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GAO’S  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates.  Listen  to  our  Podcasts. 
Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraucinet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngc1@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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